Write Clarity Smart Contracts With Zero Installations: How We Built an In-Browser Language Server Using WASM

Visual Studio Code is one of the most popular and respected code editors in use today. In fact, 71% of developers cited it as their IDE of choice in Stack Overflow’s 2021 Developer Survey. 

About a year ago, Microsoft continued to push the envelope of what’s possible in a code editor and released a web version of this editor that allows developers to start coding without installing any software. From any GitHub repository, it’s possible to press “.” to open and edit a project in VS Code for the web. Give it a try on this starter template repository.

Microsoft pioneered and implemented the Language Server Protocol (LSP), which allows developers to build support for a language once and then port that work to the many code editors that implement LSP (instead of building that support from scratch for every new code editor tool).

Because of VS Code’s popularity, and the opportunity it provides developers to start writing smart contracts for Bitcoin directly in the browser through vscode.dev or github.dev, we have been focusing on this code editor for Clarity.

The Clarity VS Code extension has helped developers write Clarity on a desktop since October 2021. The extension includes most of the features developers would expect, including linting, safety checks, step-by-step debugging, syntax-highlighting, and more. 

However, like Microsoft, we wanted to push the boundaries of software development and make a web-based editor possible for Clarity developers too. This article is about how we ported that desktop extension to the web.

The Power of Web-Based Editors

While the Clarity extension was already useful on the desktop, we thought it could do even more on the web with one-click install. No editor to configure, no code to clone, nothing to install from a package manager. VS Code for the web allows developers to open any git repository directly from the browser.

And without any installation, you immediately have access to a number of useful features, including:

Auto-Complete Functions

This feature allows you to start typing a function name, and then have the editor automatically suggest auto-completion with the documentation related to the suggestion.

Resolve Contract-Call Targeting Local Contracts

The extension auto-completes local contract calls as well.

Check Contract on Save and Display Errors Inline

When a contract is opened or saved, the extension will notify you if errors are found (syntax, unknown keyword, etc) or warnings. This helps you to ensure that you write safe and clean code.

Editor's note: if you're curious to learn more about Web3 developer tooling, watch this conversation to learn how Stacks tooling has evolved over time and what the Hiro team is currently working on.

An Overview of the Architecture of Web-Based Editors

Like many language extensions, the Clarity extension includes two main components:

• The Language Server Protocol (LSP) that helps developers write Clarity code.
• The Debug Adapter Protocol (DAP) that allows step-by-step execution of the code.

Both of these components in the existing VS Code extension rely on “Clarinet” (the Clarity SDK), which is written in Rust. The core of the LSP (the Clarity VM) was already compatible with Web Assembly (WASM), allowing us to compile it for the browser. Could we migrate our LSP desktop architecture to something new that would support browser constraints?  In doing so, we could empower developers to start writing Clarity smart contracts in under a few minutes. 

💡 WASM aims to compile programming languages, such as C or Rust, to binaries that can run in a web browser. It unlocks a lot of possibilities, considering that JavaScript used to be the only option to build applications for the web.

The Challenge: Redesigning the Extension’s Architecture

Let's get into the technical details of how we rebuilt the extension for the web. We'll be focusing on three mains aspects of the work:

• How to design the extension
• How to interact with the file system
• The impact on performance

Architecture of the Extension

A few changes had to be made to Rust code to make it WASM compatible. But the bulk of the work was to define the architecture of this web-based extension.

Thankfully, Microsoft provides an extensive list of extension examples, including lsp-web-extension-sample, which is a basic example of an LSP extension meant to run in the browser. It’s TypeScript only, but it was a solid base to get started. Another inspiration was the work done by Oso—in a blog post they described how they built a similar project (for VS Code desktop).

Based on these two resources, we started to have a pretty good idea of the architecture for our extension. Two parts are mandatory for any LSP extension:

• ‍The client (TypeScript) is the glue between the code editor and the LSP server and will inform the server when the file is opened or modified for instance.‍
• The server (TypeScript) loads the WASM code and calls it when the client needs it (for auto-completion, linting, etc).

For our purposes, we also needed to add a third part:

• The LSP logic (Rust, compiled in WASM) already existed in the desktop extension. This is where all of the logic to make the editor understand Clarity code is implemented.

There are two important things to highlight here:

• The client and the server run in their own Web Workers. This means that they run in separate threads and don't share any resources. They are only able to communicate together through the native Web Worker message events API.
• The client is built specifically for VS Code, so it has access to its APIs. However the server is editor agnostic by design, so it can't access any of its APIs.

Accessing the File System

The LSP server needs to access the File System to read and write files. Indeed, to check if a Clarity project is valid, we need to read all the clarity smart contracts of the project, parse the code and check if it is valid. The issue here, as we've discussed above, is that the server doesn't have access to the VS Code APIs, nor to the native File System API since it's running a Web Worker.

VS Code exposes a Virtual File System API (VFS), that allows the client to read and write files. So we implemented a way for the server to call the VFS through requests to the client.

💡The Virtual File System API abstracts the access to:

• the native FileSystem when running in VS Code Desktop,
• the GitHub API over https when reading files from a GitHub repo

Performance

Rust is really fast, even when compiled to WASM. We still noticed some slowdowns compared to the native version of the LSP, especially when interacting with JS functions: reading multiple files could be 20x slower before working on some optimizations.

With this in mind, we focused on high impact performance gains such as caching more information and avoiding parsing the same files multiple times.

Let’s Dive Into the Code

Let's get a step further into the technical details and look at some code.

Loading and Calling WASM

Having little experience in WASM (and in VS Code extensions), it wasn't obvious how to load or bundle the WASM code into the extension.

Our WASM package is generated with wasm-pack. It outputs a <code-rich-text>.wasm<code-rich-text> binary file and a <code-rich-text>.js<code-rich-text> to interact with it. Here is the webpack configuration to handle it.

We also use webpack to handle the URL on which the WASM file has to be fetched. It will be on <code-rich-text>localhost<code-rich-text> when working in development or test mode, and on <code-rich-text>vscode-unpkg.net<code-rich-text> in production.

💡 wasm-pack-plugin is a super handy plugin to call wasm-pack from webpack.

Then, the WASM file is loaded with <code-rich-text>fetch()<code-rich-text>. Once fetched, the content of the file is passed to the <code-rich-text>initSync()<code-rich-text> method provided by wasm-pack. See the code directly on the repository.

import { createConnection } from "vscode-languageserver/browser";
import { initSync, LspVscodeBridge } from "./clarity-lsp-browser/lsp-browser";

// ...
  initSync(await wasmModule);

  const connection = createConnection();

  const bridge = new LspVscodeBridge(
	connection.sendDiagnostics,
	connection.sendNotification,
	connection.sendRequest,
  );

Once the WASM code is loaded and initialized on the server, we can call it. Here, the idea is to have requests and notifications handlers on the TS side that forward it to the WASM package.

connection.onRequest(async (method: string, params: unknown) => {
  return bridge.onRequest(method, params);
});

That's more or less all the TypeScript part of the server is responsible for:

• loading the WASM code
• binding the TS event handlers to the Rust ones

The full code is available in these two files:

• server/src/serverBrowser.ts

• server/src/common.ts

The File System Problem

The server of an LSP for VS Code Web doesn't have access to the VS Code APIs, including the File System, which is required to read and write files.

Our solution to that was to  implement custom request handlers on the client like so:

client.onRequest("vfs/readFile", async (event: unknown) => {
  return fileArrayToString(await fs.readFile(Uri.parse(event.path)));
});

The Rust part of the server can call <code-rich-text>connection.sendRequest()<code-rich-text>, allowing it to request this <code-rich-text>vfs/readFile<code-rich-text> handler.

fn read_file(&self, path: String) -> FileAccessorResult {
let read_file_promise =
    	self.get_request_promise("vfs/readFile".into(), &WFSRequest { path });

	Box::pin(async move {
    	read_file_promise
        	.await
        	.and_then(|r| Ok(decode_from_js(r).map_err(|err| err.to_string())?))
	})
}

At first, it wasn't super straight forward to manipulate JS promises along with Rust Futures, but the <code-rich-text>wasm_bindgen_futures<code-rich-text> crate handles the hardest parts. Learn more about that crate here.

Conclusion

That’s how we ported the Clarity extension for VS Code Web! There’s still a lot of work ahead. We want to implement more LSP features like go to definition or displaying inline documentation. We also have some ideas to improve the stability and the performance of the extension.

You can try the extension directly in vscode.dev: it should automatically suggest to install the extension.