Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
3 changes: 2 additions & 1 deletion docs.json
Original file line number Diff line number Diff line change
Expand Up @@ -197,7 +197,8 @@
"docs/volumes/read-write",
"docs/volumes/info",
"docs/volumes/upload",
"docs/volumes/download"
"docs/volumes/download",
"docs/volumes/migrate"
]
},
{
Expand Down
1 change: 1 addition & 0 deletions docs/volumes.mdx
Original file line number Diff line number Diff line change
@@ -1,5 +1,6 @@
---
title: "Volumes"
description: "Persistent storage that exists independently of sandboxes and can be mounted across multiple sandboxes."
sidebarTitle: Overview
---

Expand Down
1 change: 1 addition & 0 deletions docs/volumes/download.mdx
Original file line number Diff line number Diff line change
@@ -1,5 +1,6 @@
---
title: "Download data from volume"
description: "Download files from a volume to your local filesystem with the E2B SDK."
sidebarTitle: Download data
---

Expand All @@ -7,7 +8,7 @@

<CodeGroup>
```js JavaScript & TypeScript
import fs from 'fs'

Check warning on line 11 in docs/volumes/download.mdx

View check run for this annotation

Mintlify / Mintlify Validation (e2b) - vale-spellcheck

docs/volumes/download.mdx#L11

Did you really mean 'fs'?
import { Volume } from 'e2b'

const volume = await Volume.create('my-volume')
Expand Down
1 change: 1 addition & 0 deletions docs/volumes/info.mdx
Original file line number Diff line number Diff line change
@@ -1,5 +1,6 @@
---
title: "Get information about a file or directory"
description: "Get metadata, check existence, and update permissions for files and directories in a volume."
sidebarTitle: File & directory metadata
---

Expand Down Expand Up @@ -125,7 +126,7 @@

const volume = await Volume.create('my-volume')

const fileExists = await volume.exists('/test_file.txt')

Check warning on line 129 in docs/volumes/info.mdx

View check run for this annotation

Mintlify / Mintlify Validation (e2b) - vale-spellcheck

docs/volumes/info.mdx#L129

Did you really mean 'fileExists'?
console.log(fileExists) // true or false
```
```python Python
Expand Down
1 change: 1 addition & 0 deletions docs/volumes/manage.mdx
Original file line number Diff line number Diff line change
@@ -1,5 +1,6 @@
---
title: "Managing volumes"
description: "Create, connect to, list, inspect, and destroy volumes with the E2B SDK."
---

## Create a volume
Expand Down
168 changes: 168 additions & 0 deletions docs/volumes/migrate.mdx
Original file line number Diff line number Diff line change
@@ -0,0 +1,168 @@
---
title: "Migrating data between volumes"
description: "Copy all data from one E2B volume to another by mounting both into a sandbox and using rsync."

Check warning on line 3 in docs/volumes/migrate.mdx

View check run for this annotation

Mintlify / Mintlify Validation (e2b) - vale-spellcheck

docs/volumes/migrate.mdx#L3

Did you really mean 'rsync'?
---

When you need to move data from an old volume to a new one, the simplest approach is to mount both volumes into a single sandbox and copy the files across. This page walks through a script that does exactly that: it spins up a sandbox from the **base** template, mounts a source and a destination volume, and copies all data from source to destination.

- The **source** volume must already exist. If it doesn't, the script fails.
- The **destination** volume is reused if it already exists, or created if it doesn't.

## Prerequisites

You need an E2B account with your API key set as `E2B_API_KEY` in your environment, and the E2B SDK installed.

<CodeGroup>
```bash JavaScript & TypeScript
npm install e2b
```
```bash Python
pip install e2b
```
</CodeGroup>

## The script

The script first resolves both volumes by name, connecting to the source and connecting to (or creating) the destination. It then mounts both into a sandbox and uses [rsync](https://rsync.samba.org/) to copy the data across. Reading the volume type from the token is optional, but it's a handy sanity check that you're copying between the volumes you expect.

Save this as `copy-volume.mts` (JavaScript & TypeScript) or `copy_volume.py` (Python):

<CodeGroup>
```ts JavaScript & TypeScript
import { Sandbox, Volume } from 'e2b'

const [sourceName, destName] = process.argv.slice(2)

Check warning on line 34 in docs/volumes/migrate.mdx

View check run for this annotation

Mintlify / Mintlify Validation (e2b) - vale-spellcheck

docs/volumes/migrate.mdx#L34

Did you really mean 'sourceName'?

Check warning on line 34 in docs/volumes/migrate.mdx

View check run for this annotation

Mintlify / Mintlify Validation (e2b) - vale-spellcheck

docs/volumes/migrate.mdx#L34

Did you really mean 'destName'?
if (!sourceName || !destName) {
throw new Error('Usage: copy-volume <source-volume> <dest-volume>')
}

async function findVolumeByName(name: string) {
const volumes = await Volume.list()
return volumes.find((v) => v.name === name) ?? null
}

const sourceInfo = await findVolumeByName(sourceName)

Check warning on line 44 in docs/volumes/migrate.mdx

View check run for this annotation

Mintlify / Mintlify Validation (e2b) - vale-spellcheck

docs/volumes/migrate.mdx#L44

Did you really mean 'sourceInfo'?
if (!sourceInfo) {
throw new Error(`Source volume "${sourceName}" does not exist`)
}
const sourceVolume = await Volume.connect(sourceInfo.volumeId)

Check warning on line 48 in docs/volumes/migrate.mdx

View check run for this annotation

Mintlify / Mintlify Validation (e2b) - vale-spellcheck

docs/volumes/migrate.mdx#L48

Did you really mean 'sourceVolume'?

const destInfo = await findVolumeByName(destName)

Check warning on line 50 in docs/volumes/migrate.mdx

View check run for this annotation

Mintlify / Mintlify Validation (e2b) - vale-spellcheck

docs/volumes/migrate.mdx#L50

Did you really mean 'destInfo'?
const destVolume = destInfo

Check warning on line 51 in docs/volumes/migrate.mdx

View check run for this annotation

Mintlify / Mintlify Validation (e2b) - vale-spellcheck

docs/volumes/migrate.mdx#L51

Did you really mean 'destVolume'?

Check warning on line 51 in docs/volumes/migrate.mdx

View check run for this annotation

Mintlify / Mintlify Validation (e2b) - vale-spellcheck

docs/volumes/migrate.mdx#L51

Did you really mean 'destInfo'?
? await Volume.connect(destInfo.volumeId)
: await Volume.create(destName)

function voltype(token: string): string {
// JWT payloads are base64url and often omit padding, so normalize before decoding
let segment = token.split('.')[1].replace(/-/g, '+').replace(/_/g, '/')
segment += '='.repeat((4 - (segment.length % 4)) % 4)
const payload = JSON.parse(atob(segment))
return payload.voltype
Comment thread
cursor[bot] marked this conversation as resolved.
}

console.log(`Source volume type: ${voltype(sourceVolume.token)}`)
console.log(`Destination volume type: ${voltype(destVolume.token)}`)

const sandbox = await Sandbox.create({
volumeMounts: {
'/mnt/source': sourceVolume,
'/mnt/dest': destVolume,
},
})

try {
const install = await sandbox.commands.run(
'sudo apt-get update && sudo apt-get install -y rsync',
)
if (install.exitCode !== 0) {
throw new Error(`rsync install failed: ${install.stderr}`)
}

const result = await sandbox.commands.run('rsync -a /mnt/source/ /mnt/dest/')
if (result.exitCode !== 0) {
throw new Error(`Copy failed: ${result.stderr}`)
}

const listing = await sandbox.commands.run('ls -la /mnt/dest')
console.log(listing.stdout)
} finally {
await sandbox.kill()
}
```
```python Python
import base64
import json
import sys

from e2b import Sandbox, Volume

args = sys.argv[1:]
if len(args) != 2:
raise SystemExit('Usage: copy_volume <source-volume> <dest-volume>')
source_name, dest_name = args


def find_volume_by_name(name):
return next((v for v in Volume.list() if v.name == name), None)


def voltype(token):
payload = token.split('.')[1]
payload += '=' * (-len(payload) % 4) # JWT segments omit base64 padding
return json.loads(base64.urlsafe_b64decode(payload))['voltype']


source_info = find_volume_by_name(source_name)
if source_info is None:
raise SystemExit(f'Source volume "{source_name}" does not exist')
source_volume = Volume.connect(source_info.volume_id)

dest_info = find_volume_by_name(dest_name)
dest_volume = (
Volume.connect(dest_info.volume_id) if dest_info else Volume.create(dest_name)
)

print(f'Source volume type: {voltype(source_volume.token)}')
print(f'Destination volume type: {voltype(dest_volume.token)}')

sandbox = Sandbox.create(
volume_mounts={
'/mnt/source': source_volume,
'/mnt/dest': dest_volume,
},
)

try:
install = sandbox.commands.run(
'sudo apt-get update && sudo apt-get install -y rsync',
)
if install.exit_code != 0:
raise RuntimeError(f'rsync install failed: {install.stderr}')

result = sandbox.commands.run('rsync -a /mnt/source/ /mnt/dest/')
if result.exit_code != 0:
raise RuntimeError(f'Copy failed: {result.stderr}')

listing = sandbox.commands.run('ls -la /mnt/dest')
print(listing.stdout)
finally:
sandbox.kill()
```
</CodeGroup>

## Usage

Run the script with the source and destination volume names. For example, to copy everything from `prod-data` into `prod-data-backup`:

<CodeGroup>
```bash JavaScript & TypeScript
export E2B_API_KEY=your-api-key
npx tsx copy-volume.mts prod-data prod-data-backup

Check warning on line 160 in docs/volumes/migrate.mdx

View check run for this annotation

Mintlify / Mintlify Validation (e2b) - vale-spellcheck

docs/volumes/migrate.mdx#L160

Did you really mean 'npx'?
```
```bash Python
export E2B_API_KEY=your-api-key
python copy_volume.py prod-data prod-data-backup
```
</CodeGroup>

On success it prints a listing of the destination so you can confirm the data landed. The sandbox is always shut down afterward, but both volumes (and their data) persist.
1 change: 1 addition & 0 deletions docs/volumes/mount.mdx
Original file line number Diff line number Diff line change
@@ -1,5 +1,6 @@
---
title: "Mounting volumes"
description: "Mount one or more volumes to a sandbox at custom paths when creating it."
---

You can mount one or more volumes to a sandbox when creating it. The keys of the `volumeMounts` / `volume_mounts` object are the mount paths inside the sandbox.
Expand Down
1 change: 1 addition & 0 deletions docs/volumes/read-write.mdx
Original file line number Diff line number Diff line change
@@ -1,5 +1,6 @@
---
title: "Read & write files"
description: "Read, write, list, and remove files and directories in a volume with the SDK."
sidebarTitle: Read & write
---

Expand Down
1 change: 1 addition & 0 deletions docs/volumes/upload.mdx
Original file line number Diff line number Diff line change
@@ -1,5 +1,6 @@
---
title: "Upload data to volume"
description: "Upload single files or entire directories from your local filesystem to a volume."
sidebarTitle: Upload data
---

Expand All @@ -9,7 +10,7 @@

<CodeGroup>
```js JavaScript & TypeScript
import fs from 'fs'

Check warning on line 13 in docs/volumes/upload.mdx

View check run for this annotation

Mintlify / Mintlify Validation (e2b) - vale-spellcheck

docs/volumes/upload.mdx#L13

Did you really mean 'fs'?

Check warning on line 13 in docs/volumes/upload.mdx

View check run for this annotation

Mintlify / Mintlify Validation (e2b) - vale-spellcheck

docs/volumes/upload.mdx#L13

Did you really mean 'fs'?
import { Volume } from 'e2b'

const volume = await Volume.create('my-volume')
Expand Down Expand Up @@ -41,7 +42,7 @@

const volume = await Volume.create('my-volume')

const directoryPath = '/local/dir'

Check warning on line 45 in docs/volumes/upload.mdx

View check run for this annotation

Mintlify / Mintlify Validation (e2b) - vale-spellcheck

docs/volumes/upload.mdx#L45

Did you really mean 'directoryPath'?
const files = fs.readdirSync(directoryPath)

for (const file of files) {
Expand Down
Loading