Scaling Shiny: Seamless and Effortless Distributed
Computing with mirai

April 11, 2025

Charlie Gao (Open Source - Posit, PBC)

In this talk

[ 1 ]

Shiny ExtendedTask
& mirai integration

[ 2 ]

Distributed computing
with mirai to scale Shiny

Shiny ExtendedTask

• 🧠 Unveiled by Joe Cheng (creator of Shiny) at ShinyConf 2024
• 🚀 Offload long-running tasks to another process
  → Enables intra- and inter-session responsiveness
• 🔌 mirai is the ideal async counterpart for Shiny ExtendedTask

Shiny ExtendedTask Backends

🔍 mirai vs. future: Key Differences

Aspect	mirai	future
Core purpose	Async evaluation	Parallel processing
Always non-blocking	✔	✘ (✔ future_promise)
Event-driven	✔	✘
Feels like*	reactive	observer

* Not technically equivalent, but intuitive analogies

The mirai Promise

🚀 Responsive

• ✔ Sub-millisecond response times
  
• ✔ Low overhead, zero latency

📈 Scalable

• ✔ Thousands of processes
  
• ✔ Millions of tasks

🧩 Flexible

• ✔ Mix local & remote processes
  
• ✔ Dynamic autoscaling

✨ Seamless

• ✔ Full Shiny integration
• ✔ Tight low-level coupling

Syntax

🔄 Quick Comparison

Function	mirai	future
Create async task	mirai()	future_promise()
Set up local background processes	daemons(6)	plan(multisession, workers = 6)

Scaling Solutions in Context

Before

After

Intra-Session Async

✅ Solved: Shiny ExtendedTask

• 🧠 Run in background
  Offload heavy tasks — compute, I/O, APIs

• 🚦 UI stays responsive
  No session blocking

• 📊 Serve more users
  Fewer duplicated processes, higher scalability

• 🔌 mirai efficiency
  Lowers hurdle to using ExtendedTask

How Does mirai Work?

🖥️ Local

• 📡️️ daemons(url = local_url())
  Opens a local IPC socket
  e.g. "abstract://85de9c..."

• 🛰️️ launch_local()
  Launches 1 local daemon

• daemons(6)
  Sets and launches 6 local daemons

🌐 Distributed

• 📡️️ daemons(url = host_url())
  Opens a TCP network socket
  e.g. "tcp://myhostname:40491"

• launch_remote()
  Shows shell command to connect

• 🛰️launch_remote(remote = ssh_config("ssh://192.168.0.10"))
  Launches 1 remote daemon

• daemons(6, url = host_url(), remote = ssh_config("ssh://192.168.0.10"))
  Sets and launches 6 remote daemons

Distributed Computing over Tunnelled SSH

🔐 SSH tunnelling allows the remote machine to connect back to your network socket as if it were local to that machine

🥇 Step 1: Create the remote launch configuration

cfg <- ssh_config("ssh://192.168.0.10", tunnel = TRUE)

🥈 Step 2: Set daemons using a local TCP URL

daemons(n = 2, url = local_url(tcp = TRUE), remote = cfg)

🥉 Step 3: Launch and scale (with optional auto-scaling)

launch_remote(n = 2, remote = cfg)
launch_remote(n = 2, remote = cfg, idletime = 1000 * 60)

Thanks

Scaling Shiny: Seamless and Effortless Distributed Computing with mirai

https://github.com/r-lib/mirai