DFINITY Foundation

Internet Computer Association

Build long term R&D efforts for General integration (from which multiple projects will emerge)

## 1. Objective
The objective of this initiative is to integrate the Internet Computer with outside systems. We have identified multiple classes of external systems of interest in this domain: The Web 2.0, i.e., HTTP(S) and other standard protocols, other blockchains, and end user devices. As can be observed, these are quite different integrations collectively put together in this motion proposal. Ultimately, the goal of integrating the Internet Computer with such systems is to provide greater value to both the IC and the systems it integrates with.

## 2. Background
A blockchain is per se isolated from other systems outside of it. The standard defined interaction pattern with the outside world is that of users interacting with the blockchain by submitting transactions (or queries) and obtaining responses. The blockchain nodes cannot arbitrarily call out to servers on the Internet to retrieve data or send transactions to smart contracts on other blockchains. Replicas directly obtaining data from external data sources would likely break the consensus layer because different replicas would receive different responses (e.g., due to timestamps, unique ids, or changed data due to differences in the times when the requests were made). Blockchains solve the problem of providing external data to smart contracts through so-called oracles, the realization of which spans a spectrum from centralized to decentralized architectures. In every instance, though, external parties, the oracles, are required to provide data, thus complicating the overall trust assumptions of the smart contracts and increasing operational cost.
Making transactions to other blockchains is even more difficult in that transactions need to be signed and in the trust model of blockchains, network nodes cannot securely hold a signature private key, so threshold cryptography or trusted entities need to be used. Most integrations we see as of now are based on wrapping tokens of other chains through (centralized) bridges to enable DeFi use cases. General invocations between smart contracts on different chains are harder, but there are efforts of standardizing protocols for this purpose.
A quite different problem domain is the currently widespread use of proprietary centralized public cloud services through end users, e.g., through mobile apps that make use of these cloud services through easy-to-use SDKs provided by the cloud vendors. A good example for this are SDKs that offer cloud storage services to apps. There are no SDKs yet that would offer the functionality of the Internet Computer to mobile app developers in an easy-to-use manner. This should be addressed as part of this integration proposal. In contrast to integrating the IC with external systems, this is rather a use case where IC functionality is made available to end users via app SDKs, much like this is prominently done with cloud services these days.

## 3. Why this is important
The Internet Computer as a standalone system can already provide great value by offering a platform for running smart contracts at web-speed and even serving the frontend from the blockchain. However, many interesting types of smart contracts, such as decentralized finance (DeFi) smart contracts or decentralized insurance smart contracts, require external data in order to function. Such smart contracts are referred to as *hybrid smart contracts* as they, in addition to what is available on chain, also rely on external data for their logic. Thus, a solution that allows canisters to directly make HTTP requests to outside servers, an integration with existing oracle networks, or a proprietary oracle network is crucial for enabling vital use cases on the Internet Computer.
The integration with other blockchains can be considered the next level of integration with external systems and enables further use cases: If we would have broad inter-chain connectivity among the top blockchains, smart contracts on all chains could interact with each other, thereby opening up a plethora of new use cases and leveraging network effects. An intermediate step towards this would be a broad availability of wrapped tokens of other blockchains on the Internet Computer. Besides the availability of external data, the availability of tokens of other chains is crucial for certain exciting high-value use cases such as decentralized exchanges (DEXs).

## 4. Topics under this project
We currently see the following topics of interest under the umbrella of this initiative:
* **HTTP(S) support for canisters**
An important integration for the Internet Computer is the one with the current Web 2.0 infrastructure by allowing canisters to make HTTP(S) calls to servers on the Internet. This is a non-trivial problem in multiple ways:
  * The replicas of the IC each need to make the same call and obtain consensus over the result. However, results can differ, e.g., because of unique identifiers, timestamps, or slightly deviating numerical values, e.g., for quotes. Those differences need to be accounted for when bringing the HTTP call results through IC consensus.
  * Calls that change the state in the invoked external service, i.e., are not idempotent, pose another interesting challenge.
Having an HTTP call mechanism enables a wide range of applications, e.g., oracle services directly integrated into the Internet Computer in a trustless manner, user notifications, and anything else that requires communication with servers on the Internet.
* **Support of further protocols for canisters**
Besides HTTP, we intend to support other relevant protocols to integrate with Web 2.0 systems. To this end, we need to further investigate which protocols should be considered. Community input about which protocols to consider and the use cases this would solve will be greatly appreciated.
* **Oracles**
An important part of this multi-year integration strategy are oracles. Oracles are crucial for providing data such as stock quotes, cryptocurrency, or any other relevant market data for DeFi, weather data for decentralized insurance applications, or any sort of interesting data feeds to smart contracts on the IC. The functionality of canisters making HTTP calls is closely related to oracles, however in a first iteration, HTTP calls will not provide all functionality the most powerful oracle services of today. Thus, we want to also investigate an integration with existing oracle services, e.g., Chainlink, to leverage existing oracle ecosystems that are readily available and immediately benefit from the accompanying network effects.
* **Blockchain integrations**
Particularly the domain of DeFi applications, e.g., decentralized exchanges (DEXs), integrations with other blockchains and specifically their respective native cryptocurrencies, besides Bitcoin and Ethereum, is of relevance. The Foundation does not have the resources to integrate with all other blockchains of interest, and therefore considers, as part of this motion proposal, to provide a blueprint or framework to enable the community to implement such integrations on their own or to use upcoming integration services. This will open up the Internet Computer for a plethora of exciting DeFi use cases that are not yet possible in their entirety.
* **End user device integration**: Cloud SDK alternatives
Another quite different topic in the area of this motion proposal are blockchain alternatives to the cloud integration SDKs for mobile operating systems with API compatibility. Such cloud SDKs are currently available on the iOS and Android mobile platforms and allow users to, for example, store data on legacy cloud systems. Providing an API-compatible alternative to those SDKs would allow users to (almost transparently) use the same APIs (or a subset thereof) to perform the same functionality on the blockchain instead of the traditional cloud. A specific use case here would be storage of data. This could help users move from the Web 2.0 world of proprietary cloud stacks to an open blockchain world with many of their daily computing needs. This is quite different to the other integrations as it is rather a use case of the Internet Computer in the context of replacing traditional cloud computing technology.

## 5. Key milestones (and technical solution if known)
As this is a multi-year R&D initiative, we can currently not yet provide the concrete milestones for the individual features. Each feature will require milestones for the finalized system design, a PoC or MVP implementation, and further milestones for specific core functionalities. We plan to take an agile approach for the initiative in that we prioritize the functionalities, together with the community, and provide implementations early on to provide best possible value to the community. This will imply an agile approach of delivering MVPs initially that provide basic 
functionality and later expanding the functionality based on what is most needed.

## 6. Discussion leads
Dieter Sommer, Thomas Locher, Timo Hanke 

## 7. Why the DFINITY Foundation should make this a long-running R&D project
The features addressed in this initiative motion proposal cover a broad spectrum of functionality and require substantial R&D efforts to bring them to the market. Basic functionality of the features may be relatively fast to implement, but more advanced functionality will require time for the underlying research efforts to develop feasible solutions to the problems. Based on those, system designs and implementations thereof need to be produced. Considering the expected breath and depth of this initiative, we think it can only be tackled through a multi-year initiative that is operated in an agile manner.

## 8. Skills and Expertise necessary to accomplish this (maybe teams?)
Due to the complexity of the initiative, we require teams with a broad selection of skills as outlined next:
* System design
* System level software engineering
* Algorithms, complexity
* Probability theory
* Cryptography
* Technologies behind different blockchains
* Deep understanding of Internet Computer consensus
* Oracle technologies
* Mobile cloud SDKs
* API design

At least the following teams are likely required:
* Research
* Networking
* Consensus
* Message Routing
* Execution
* NNS
* Security
* SDK

## 9. Open research questions
* Consensus protocol extension for agreeing on HTTP(S) responses
* Response aggregation in consensus to realize oracle functionality without oracles
* Architecture for integrating HTTP(S) calling with the IC protocol stack and particularly the consensus layer
* Threshold cryptography
* Algorithm research required for the integrations (e.g., management of connections to other blockchain nodes, handling of incoming blockchain blocks, block processing, protocol between IC components to request and receive blocks)
* Analogous questions for different protocols to integrate (e.g., streaming protocols)
* Evaluation of oracle networks and definition of integration architecture
* General approach / blueprint to integrating with other blockchains
* Security modeling and proofs

## 10. Examples how the community can contribute to the project
* Review proposals
* Provide feedback on the dev forum
* Submit NNS proposals
* Review technical designs
* Contribute to technical designs
* Review code
* Contribute code
* Participate in workshops on the initiative or its features
* Help us prioritize for the best benefit of the community

## 11. What we are asking the community
* Review comments, ask questions, give feedback on developer forum: https://forum.dfinity.org/t/long-term-r-d-general-integration-proposal/9383/4
* Vote accept or reject on NNS Motion


Long Term R&D: General Integration