TWIL: Deploying Node.js to Azure App Service with GitHub Actions

TWIL (This week I learned) all about deploying a Node.js server to Azure App Service using GitHub Actions. 👨‍💻 On Wednesday, it was King’s Day in the Netherlands where I live. 👑 And while most people were out on the tiles dressed in orange, 🧡 I was buried in code from morning till dawn (work that one out) learning new skills required for the deployment of the token bridge server for Schnoodle ĐAO which will be a critical part of the release of SchnoodleV9, enabling cross-chain interoperability for SNOOD across all EVM-compatible chains. ⛓️

Last month, Schnoodle ĐAO published this story on the architecture of our Fully Autonomous Bridge (FAB). 🌉

The underlying technology of FAB is the Polybridge Infinichain Meshnet Protocol, or PIMP for short. 😎 And the focus of the major GitHub commit below is the server part of the architecture (in the diagram above) which is a Node.js server that must be deployed to Azure App Service.

Refactor token bridge server to be deployable to Azure App Service

For the deployment, I chose to use GitHub Actions which means that every commit results in an automatic deployment. This is known as CI/CD (continuous integration and continuous delivery), something I’m generally a huge fan of in software development as it encourages good DevOps software practices such as automated build and release pipelines, test automation, and incremental changes (failures are more easily identifiable). ♾️

While our Node.js token bridge server is not overly complex, it includes some features that presented a few obstacles that I had to find solutions for along the way, all while learning the skills needed. I was effectively a noob trying to go from zero 💀 to hero 🦸‍♂️ in 24 hours.

The first step, using these instructions, was to create an App Service in Azure with a deployment slot for testing, and connect it to our GitHub repo in the Deployment Center. Easy-peasy. 🍋

Azure App Service — Deployment Center

This generated a GitHub Actions workflow file (like the one below) with some default steps. One of those is npm run test. And this is the commit that was automatically created by this step to connect Azure App Service to GitHub using this workflow file. 👨‍💻

The server uses Fastify, which is a fast and low overhead web framework. 🌐 This allows the server to be called over the Internet from the frontend DApp (Schnoodle X) to perform the centralised activities required for the token bridge. 🧐 To test this functionality, I had to add Node Tap (by isaacs) which is a Test-Anything-Protocol (TAP) library for JavaScript. For now, only basic tests have been added as the most important testing is in the smart contract unit tests which are comprehensive and can be found here.

Schnoodle uses what is known as a monorepo. This is where all the code for all parts of the system is in a single GitHub repository. This is a trend spearheaded by some of the biggest tech companies such as Google, Facebook, Microsoft, Uber, Airbnb, and Twitter. 🤯

This has many advantages including having all the code in one place. However, this presented three problems for me when deploying the Node.js server to Azure App Service.

Problem 1: Folder structure

The server code is not in the root folder; it’s in a subfolder. 📂 This resulted in various build and deploy errors that had me scratching my head for hours, 🤔 and I found various hack solutions online that all seemed a bit ugly. 🧟‍♂️ In the end, there were three main changes I had to make to get it to work in a clean way:

1. One change was to set the default working-directory in the workflow file so that the build process used the correct folder to build per this solution on Stack Overflow.
2. Another change was to add PROJECT to the App Service application settings, and assign to it the path of the project in the repo. This is so that the server code would be deployed in wwwroot and not a subfolder of it. Otherwise, calls over the Internet would not work as expected. 🚫 This issue relates to Kudu, which is the engine behind a number of features in Azure App Service related to source control based deployment. Once my searches led me to a similar issue raised on GitHub about Kudu, that led me to the documented solution here. 💪 Sometimes, the solutions are well documented, but it takes someone to experience the problem one would have without the correct code in place for the Google search algorithms to lead you to those solutions.
3. Finally, due to a nondescript error I was getting, one more change was to set SCM_DO_BUILD_DURING_DEPLOYMENT in application settings to false. A rather annoying error, and some discussions I found online alluded to this being a GitHub Actions bug. 🪲 In any case, it was solved as per this article that I found.

Problem 2: Smart contract build files

When Truffle generates the smart contract build files during compilation, they are outputted to a folder, but both the Node.js server and the frontend React app must be able to access these files. With a monorepo, this is easy using relative paths in a local dev environment. 👨‍💻 However, for server deployment, this doesn’t work as the deployed files do not contain all the files in the monorepo; only the files required for the particular app within the monorepo, such as the Node.js server. 🙀

To solve this, I had to reference the folder containing the smart contract build files using symbolic links, or symlinks. 🔗 I’ve used symlinks many times before for other reasons, but never in a GitHub repo. I found a good discussion on symlinks in GitHub here. I then resolved it by adding two symlinks. One for the Node.js server, and one for the React app. Problem solved! ✅ A useful article on how to add symlinks in Windows can be found here.

Problem 3: Large number of files in deployment

Due to the amount of files in node_modules (a rather infamous “problem”), I received a warning during the build:

⚠️ “There are over 10,000 files in this artifact, consider creating an archive before upload to improve the upload performance.”

The clue was in the message, and I followed the helpful instructions in this Stack Overflow answer which advised to zip and unzip the artifacts during deployment.

Unfortunately, due to the unusual combination of also using the working-directory option, this didn’t go as well as I’d hoped. 😿

Deploy error using zipped artifact together with working-directory option

I spent hours searching for a solution, until I wrote a question on Stack Overflow myself to seek an answer from someone. And then it dawned on me 💡 (after scrutinising the logs for some time). Unzip to the same folder I zipped from. This worked! 🎉 I posted the solution as an answer to my own question.

Secrets!

Finally, and I was now very much burning the midnight oil at this point, 🕯️ I wanted to solve the problem of storing and accessing secrets 🤭 such as the private key of the EOA that interacts with the target chain in any bridge operation. During local development, settings and secrets can be stored in .env files accessed using dotenv.

Thankfully, I already had an Azure Key Vault set up from developing the NFT platform where we store secrets such as the Pinata API key, and the keys for Azure Cosmos DB and the Azure storage account. 🦾 I now just had to add the application secrets there, and access them from the Node.js server by following the instructions here (using Azure Managed Identities).

Success!

And voilà. The server is now deployed. 💥 Not only is this a great step for SNOOD as we get closer to the release of the most secure and progressive token bridge in the crypto space, this was also a very fulfilling and enjoyable personal technological journey for me. Learning new skills keeps our minds fresh and our brains healthy. 🧠

Never stop. Never give up. Be a winner! 🥇 Even if that means geeking it up for nearly 24 hours nonstop on a national holiday!

Geeking it up I said, not gimping it up!

Feel free to test the server is alive via this link. 🕺

{“status”:”ok”}