Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Implement VM snapshots #379

Merged
merged 15 commits into from
Jul 14, 2023
Merged

Implement VM snapshots #379

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
15 commits
Select commit Hold shift + click to select a range
709c635
Problem: Instances loose data between nodes, so we need a way to hand…
nesitor Jul 13, 2023
ca9350e
Fix: Added some refactor for snapshots and handled some errors.
nesitor Jul 13, 2023
2285bd9
Fix: Split snapshot class in two.
nesitor Jul 13, 2023
ee5b7f9
Fix: Make it work well with threads and asyncio.
nesitor Jul 13, 2023
22be027
Fix: Added code improvements with threading and fixed PR comments.
nesitor Jul 14, 2023
1999a9e
Fix: Fixed check_disk method error.
nesitor Jul 14, 2023
7c3231d
Fix: Fixed more snapshot errors with multithreading
nesitor Jul 14, 2023
b9cc7ae
Fix: CI for Ubuntu was failing too often because of apt/dpkg locks (#…
odesenfans Jul 14, 2023
6580baf
Create Ubuntu rootfs (#370)
nesitor Jul 14, 2023
08b1605
Fix: Use SDK version 0.7.0 (#371)
hoh Jul 14, 2023
5deb1ed
Support BTRFS filesystem for instances (#373)
nesitor Jul 14, 2023
b39137b
Feature: Delete old snapshots after do a new one.
Jul 14, 2023
e2d9640
add dependency to python3-schedule
odesenfans Jul 14, 2023
fb381a3
Fix latest_snapshot attribute error
odesenfans Jul 14, 2023
e09ac3b
default snapshot frequency 1 hour
odesenfans Jul 14, 2023
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
8 changes: 3 additions & 5 deletions .github/workflows/test-on-droplet-ubuntu-22.04.yml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -33,7 +33,7 @@ jobs:
run: |
doctl compute droplet create \
--image ubuntu-22-04-x64 \
--size c-2 \
--size c-4 \
--region fra1 \
--vpc-uuid 992896c8-c089-4da3-9288-f81e28c095a4 \
--enable-ipv6 \
Expand All @@ -55,11 +55,9 @@ jobs:
export DROPLET_IPV4="$(doctl compute droplet get aleph-vm-ci-ubuntu-22-04 --output json | ./.github/scripts/extract_droplet_ipv4.py)"
ssh-keyscan -H ${DROPLET_IPV4} > ~/.ssh/known_hosts

# Ubuntu droplets run upgrades at boot
# Ubuntu droplets run upgrades at boot, which locks apt-get
sleep 30
until ! ssh root@${DROPLET_IPV4} "lslocks --json | grep /var/lib/dpkg/lock" > /dev/null; do sleep 1; echo "Waiting for dpkg lock..."; done

ssh root@${DROPLET_IPV4} DEBIAN_FRONTEND=noninteractive "apt-get update"
until ! ssh root@${DROPLET_IPV4} "apt-get update" > /dev/null; do sleep 1; echo "Waiting for apt/dpkg lock..."; done
ssh root@${DROPLET_IPV4} DEBIAN_FRONTEND=noninteractive "apt-get upgrade -y"
ssh root@${DROPLET_IPV4} DEBIAN_FRONTEND=noninteractive "apt-get install -y docker.io apparmor-profiles"
ssh root@${DROPLET_IPV4} "docker run -d -p 127.0.0.1:4021:4021/tcp --restart=always --name vm-connector alephim/vm-connector:alpha"
Expand Down
2 changes: 1 addition & 1 deletion kernels/microvm-kernel-x86_64-5.10.config
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -2113,7 +2113,7 @@ CONFIG_FS_MBCACHE=y
# CONFIG_JFS_FS is not set
# CONFIG_XFS_FS is not set
# CONFIG_GFS2_FS is not set
# CONFIG_BTRFS_FS is not set
CONFIG_BTRFS_FS=y
# CONFIG_NILFS2_FS is not set
# CONFIG_F2FS_FS is not set
# CONFIG_FS_DAX is not set
Expand Down
2 changes: 1 addition & 1 deletion packaging/aleph-vm/DEBIAN/control
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -3,6 +3,6 @@ Version: 0.1.8
Architecture: all
Maintainer: Aleph.im
Description: Aleph.im VM execution engine
Depends: python3,python3-pip,python3-aiohttp,python3-msgpack,python3-aiodns,python3-alembic,python3-sqlalchemy,python3-setproctitle,redis,python3-aioredis,python3-psutil,sudo,acl,curl,systemd-container,squashfs-tools,debootstrap,python3-packaging,python3-cpuinfo,python3-nftables,python3-jsonschema,cloud-image-utils,ndppd,python3-yaml,python3-dotenv
Depends: python3,python3-pip,python3-aiohttp,python3-msgpack,python3-aiodns,python3-alembic,python3-sqlalchemy,python3-setproctitle,redis,python3-aioredis,python3-psutil,sudo,acl,curl,systemd-container,squashfs-tools,debootstrap,python3-packaging,python3-cpuinfo,python3-nftables,python3-jsonschema,cloud-image-utils,ndppd,python3-yaml,python3-dotenv,python3-schedule
Section: aleph-im
Priority: Extra
2 changes: 1 addition & 1 deletion runtimes/aleph-debian-11-python/create_disk_image.sh
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -34,7 +34,7 @@ apt-get install -y --no-install-recommends --no-install-suggests \
pip3 install 'fastapi~=0.95.1'

echo "Pip installing aleph-client"
pip3 install 'aleph-sdk-python==0.7.0b1'
pip3 install 'aleph-sdk-python==0.7.0'

# Compile all Python bytecode
python3 -m compileall -f /usr/local/lib/python3.9
Expand Down
59 changes: 0 additions & 59 deletions runtimes/instance-debian-rootfs/Dockerfile
View file
Open in desktop

This file was deleted.

33 changes: 0 additions & 33 deletions runtimes/instance-debian-rootfs/create-debian-disk.sh
View file
Open in desktop

This file was deleted.

37 changes: 0 additions & 37 deletions runtimes/instance-debian-rootfs/init0.sh
View file
Open in desktop

This file was deleted.

Loading