DFINITY Foundation

Internet Computer Association

Build long term R&D efforts for Canister Migration (from which multiple projects will emerge)



## Canister Migration

The Internet Computer (IC) blockchain is composed of multiple subnet blockchains that can host dapps consisting of canister smart contracts. With the help of chain key cryptography those canisters can transparently communicate with each other by sending messages back and forth irrespective of the subnet they are hosted by. The IC can scale out its computational capacity by adding more subnets. However, currently there is no easy way to move a canister smart contract from one subnet to another for load balancing and co-location purposes.

### Objective

This motion proposal is about efforts related to enabling the migration of canister smart contracts between different subnet blockchains of the IC. The objective of canister migration is to provide a tool for load balancing on the IC by allowing developers to move canisters or groups of canisters from one subnet to another. For example, one could envision moving a popular canister away from an otherwise heavily loaded subnet so that the popular dapp can meet its service level objectives again.

### Why is this important

Currently, the only way to move a canister smart contract from one subnet to another is for the canister developer to implement an API that allows downloading the state of the canister, download the state via this API, create a new canister, upload the state to the new canister, and delete the old canister. Obviously this has quite a few downsides, including but not limited to:

1. Every user of a dapp that is migrated this way would need to trust the entity who downloads and uploads the state to not modify the state before uploading it again.
2. Currently canisters address other canisters using the canister ID. A newly created canister will have a new ID, meaning that currently all canisters that want to talk to the migrated canister will have to be updated to talk to the right canister.
3. One needs to take care that the state of the canister to be migrated is not modified after its state has been downloaded and/or to implement measures to keep the old and the new canister in sync.
4. There is no easy way for users who were using the old canister to find the new canister after a canister has been migrated this way.
5. When users interact with canisters using Internet identity (II) they do so via a principal identifier that is, among others, specific to the canister they interact with. If the canister ID changes upon migration, a canister that uses II will not be able to link interactions of the same user with the old canister and the new canister anymore.

Obviously this is quite a complicated process that requires a lot of effort and where things may easily go wrong. The high-level goal of this project is to make things less cumbersome for dapp developers while at the same time also reducing the required trust assumptions, and in doing so making it easier to scale dapps.

With a growing number of subnets and canisters, a higher degree of automation is required as well, as load balancing tasks can no longer be handled manually.

### Key milestones

Four main milestones are currently envisioned in this project. The first milestone is mostly about improving existing tooling and is less research heavy, while the remaining milestones still require quite some research regarding how a feature like this could be realized at all, and how things can be done so that they are operationally convenient for dapp developers and users. The milestones are as follows:

1. Extend the tooling to make it easier for dapp developers to download the state of their canister and upload it to a different subnet. Note that the first milestone is just about making what is already possible more convenient for developers.
2. Enable migration of canisters via a proposal in a way transparent to dapp developers in the sense that the other canisters that sent messages to the migrating canister before migration will not have to be adapted to send to the new canister. Do note, however, that there will likely be some downtime of the canister during migration and there will be potentially increased latency when talking to the migrated canister via canister-to-canister calls due to the canister migrating to another subnet.
3. Enable migration of canister groups so that dapps that are composed of multiple canisters can migrate as a whole.
4. Further automate canister migration so that canisters (or groups of canisters) can be automatically migrated based on various parameters of the system like the load on a subnet, for example.

Input regarding prioritization or organization of the milestones from the community is highly appreciated.


### Why the DFINITY Foundation should make this a long-running R&D project

This project is important because currently dapp developers may hit scalability issues because of their colocation with other dapps on the same subnet. Without working on this motion proposal dapps may not be able to meet their service level objectives anymore as soon as subnets start to fill up, making it impossible for an application to grow without becoming spread across multiple subnets and incurring extra latency.

### Skills and Expertise necessary to accomplish this

Realizing canister migration will require to develop new schemes and protocols that will enable authentically transferring canisters from one subnet to another and to adapt the existing schemes and protocols to guarantee that the migration process can be done in a way transparent to canisters and users. Clearly this is a broad R&D effort with many open questions requiring the involvement of many teams all across the IC stack that will take years to complete. In addition it also requires broad input from the community to guarantee that the resulting solution meets the needs of both canister developers and users.

### Additional open research questions

While the goals outlined in the previous sections already directly pose many of the research questions that need to be answered, there are also some questions that are less explicit. In this section additional research questions are listed:

* Currently canisters are addressed by their ID and routing happens via the so-called routing table. The routing table maps from ranges of canister IDs to the hosting subnets. Migrating individual canisters will lead to fragmentation of the routing table. It needs to be answered as to how to avoid fragmentation of the routing table while allowing canisters to migrate. Initial directions to answer this question include the introduction of a DNS-like system that would enable a migrated canister to take a new canister ID while its alias remains constant and can be used to look up the current ID of the canister.
* How can migration be done in a way compatible with II, i.e., so that each user interacting with a canister via II before migration will be identified as the same user after migration. As hinted at before, not changing the canister ID upon migration is one way to achieve this. However, not changing the canister ID is in conflict with the previous research question. So the goal of this question is to answer how to get the best of both worlds, i.e., potentially allowing to change the canister ID while also not breaking II.
* What is the downtime of a canister that is migrated and how can the downtime upon migration be minimized?
* Does migration require putting canisters into the stopped state before migration or can this be avoided?
* What are suitable canister placement and re-allocation algorithms?

### Examples where community can integrate into project

As already mentioned before, in the initial phase of this motion input regarding refining the scope and priorities of this project from the community is highly appreciated. In addition many technical discussions with the community are anticipated as the motion and research and development of potential technical solutions to address the goals of this proposal move forward.

### What we are asking the community

* Review comments, ask questions, give feedback
* Vote accept or reject on NNS Motion
* Participate in technical discussions as the motion moves forward


Developer forum: https://forum.dfinity.org/t/long-term-r-d-canister-migration-proposal/9403/1 





Long Term R&D: Canister Migration