iSEE singlecellexperiment GUI app on shinyapps not working, Troubleshooting

Question

[Gabriel]

Hello, I have a simply singlecellexperiment object, and I have managed to make a shiny app using the iSEE package http://www.bioconductor.org/packages/release/bioc/html/iSEE.html

I made an app.R file to run the code that returns a shiny app, then I use runApp as you are supposed to do in the end of such a file, I am trying to upload it or deploy it and it's not working

library(BiocManager)
library(rsconnect)
options(repos = BiocManager::repositories())
library(shiny)
library(iSEE)
sce = readRDS("sce.RDS")
app <- iSEE(appTitle="test",sce)
if(interactive()){shiny::runApp(app)}

I get this error:

Error in shiny::runApp(app) : Can't call runApp() from within runApp(). If your application code contains runApp(), please remove it.

If I put just

iSEE(appTitle="test",sce)

It is able to load and deploy on my shiny apps account and it says "running", however I now get an "out of memory" error on the shinyapps server:

2020-10-08T09:12:21.153674+00:00 shinyapps[2969507]:     aperm, apply, rowsum
2020-10-08T09:12:21.153674+00:00 shinyapps[2969507]: 
2020-10-08T09:12:21.496053+00:00 shinyapps[2969507]: Loading required package: SingleCellExperiment
2020-10-08T09:12:26.698874+00:00 shinyapps[system]: Out of memory!

It's nice that this library works in my R session and local browser, but if I am unable to deploy the GUI app online, it is pretty useless.

I found out that the free version's bundle size is limited to 1GB ( https://docs.rstudio.com/shinyapps.io/applications.html ), and my app sce object is just around 0.9 GB, I will try to reduce the size and try again.

> format(object.size(sce), units = "GB")

[1] "0.9 Gb"

I reduced the memory size of my sce object to 0.6 GB but the error is still happening. It seems the memory size of "SingleCellExperiment" package is way too large and I suspect it imports some other sce-datasets that are very large, is there any way to route this issue?

[Jean-Karim Heriche]

Free services are often going to be limiting in one way or the other. If you have specific requirements and don't want to pay for a hosting service, you should consider self-hosting. You can deploy shiny apps using ShinyProxy. If you expect users to upload their data, note that there's a default limit to the size of files that can be uploaded to a shiny app. You can increase this limit in your app with options(shiny.maxRequestSize=X) where X is the new limit in bytes.

Answer 1

[kevinrue67]

Hi @Gabriel,

For your first issue with runApp(), I can't find exactly the documentation on shinyapp.io right now, but checking our own apps, we never explicitly call runApp(), instead we finish the script with iSEE(sce, tour = tour, initialPanels = initialPanels, voice = TRUE)

See examples within https://github.com/iSEE/iSEE2018/tree/master/tours

For other apps, I finish them with a call to

shinyApp(ui = ui, server = server)

See https://github.com/kevinrue/magick-profile/blob/master/magick-profile/app.R

I imagine that shinyapp.io runs runApp() for you.

Secondly, yes, that looks certainly like a memory issue. I ran into the same challenges when setting up an app to run on shinyapps.io for the RStudio contest.

I had to downsample the dataset quite a lot to fit within the 1GB free memory allowance. This limit is quite restrictive. Checking back, I downsampled aggressively the demo object to ~200MB in this session: https://rstudio.cloud/project/230765 However, I never managed to identify a "maximum size" that the 1GB limit can handle, it might be more than 200MB, but probably not that much more, most likely less than 500 MB.

You have to consider that the 1GB limit applies to the whole session on shinyapps.io, this includes your original SCE object, a copy that the iSEE app uses internally to cache some information necessary at runtime, and then each panel in the interface also has a data.frame for the table or plot that is diplayed. (I'm sure I forget other objects that get created under the hood, but you get the picture that all of those objects add up on top of your original SCE object and count toward the 1 GB limit).

Unfortunately, all I can recommend is either to set up your own Shiny server (free or paid), or to pay for a shinyapps.io account with more memory. RStudio offers a number of paid solutions, but I don't want to make any recommendation toward either free or paid solutions, that should be up to you, depending on your needs, budget, IT support, and existing infrastructure.

On a related note, there is some support for iSEE to support arbitrarily large datasets in a memory efficient way using HDF5, so that entire assays are not stored in memory: https://bioconductor.org/packages/devel/bioc/vignettes/iSEE/inst/doc/bigdata.html

I hope this helps!

Kevin

[Jean-Karim Heriche]

An open source alternative to RStudio server is ShinyProxy.