The Firmware Development Toolchain of the Future

by | Jan 11, 2022

[Estimated Time to Read: 6 Minutes]

There is an intellectual joy that comes from developing and debugging firmware code. The smell of the solder, the hum of the power supply, solving the graphic puzzle displayed on the oscilloscope – it’s true engineering! There’s also a long list of annoyances and inefficiencies associated with firmware development that our colleagues over in web and software don’t have to worry about. That said, the world of firmware development is quickly changing due to pressures of distributed teams, complex IoT products, and ever-decreasing expectations of development time. 

In his Ioterra PDC workshop, Akbar Dhanaliwala, CEO of LagerData, paints a picture of what the future of firmware development will look like and highlights three key trends that will change the industry over the next few years. Akbar also highlights a few techniques and tools that modern firmware developers should be familiar with.

Watch Akbar’s full 30 minute Ioterra PDC talk here

The Current Headaches of Firmware Development

Let’s start by quickly summarizing the primary aspects that make modern firmware development challenging.

 

Modern teams are distributed

The thing about firmware is it is intricately tied to the hardware. In the past this was fine; traditionally you’d have your hardware sitting on your desk connected via UART or JTAG to your computer. However with the rise of remote development culture, most firmware teams are not colocated and most developers do not have the advantage of the hardware sitting on the desk in front of them.

 

Firmware files have become lengthy

A decade ago firmware files were concise; hundreds of lines of code, maybe thousands. Now with our increasingly beefy microprocessors and increasingly sophisticated products, firmware files may be millions of lines long. The manual code review process that was typical for firmware development is no longer feasible for these massive files.

 

Toolchains for firmware are simply old

Some of the main tools of the firmware developer such as JTAG and UART have remained largely unchanged for 30 or 40 years. They are tried and tested, sure, but they can be clunky to integrate into modern software processes.

 

Other firmware challenges

Other considerations that are unique to the firmware developer are lack of standardization among communication protocols, real-time operating requirements, a need to minimize power consumption, and the need to implement adequate security, identity, and authentication parameters.

“None of my projects have local tool chains. I put everything immediately into a Docker image. Then it’s easy for me to share it and maintain it. There’s no risk of updating python locally and breaking my entire toolchain, or updating GCC, or something like that. It’s gonna always work. I think [firmware containerization] is going to definitely be one thing that we’re going to start seeing more and more in the future of IoT.”

– Akbar Dhanaliwala

Firmware development teams have become increasingly distributed over the past decade, it’s no longer feasible for all team members to have the device under test sitting in front of them. Photo courtesy of LagerData.

Trends that will Address these Pain Points

 

Trend #1 – Containerization

Many people still consider containerization to primarily be a tool for software development. But there are a plethora of reasons that modern teams should be using containerization embedded firmware development.

In short, containerization allows the developer to create the built environment once and subsequently clone it and share it such that the same codebase can be executed in the cloud, on a different computer, on a different microprocessor, wherever. The advantage of containers relatieve to say, virtual machines, is that they are significantly more lightweight.

Docker is the major player for containerization. For firmware developers who are not familiar with Docker, it is worth taking a look. Akbar mentions that their team never stores firmware code locally at LagerData. It all goes into their cloud-based Docker container immediately so that everyone on their team can access it and execute it.

In the future when you start at a new company as a firmware developer, there will be almost zero time required getting your computer set up with their custom build chain. The company will simply give you access to their repo, you’ll do a docker pull, and you’ll be good to code! 

Containerization is a well known approach in the world of software development, but it hasn’t yet made much inroads into embedded development. That will likely change over the next few years. A containerized embedded stack is depicted above. Photo courtesy of LagerData.

Trend #2 – Hardware-in-the-Loop Testing

More firmware testing is sorely needed for modern hardware products, especially complex and connected IoT products. Firmware testing is significantly less thorough than software of comparable complexity. In the past with shorter firmware filesets and simpler devices, testing would be done manually from a checklist of tests to perform. However with modern, connected devices, unit testing and integration testing on firmware needs to be more comprehensive. 

There’s a number of tools that perform continuous integration and continuous development (CI/CD) for software, such as CircleCI, that can be adapted for use with firmware. The trick is properly hooking up your physical hardware to an always-on, always-accessible server that can be running real-time tests. Getting this right remains a bit unwieldy and there are a number of companies, including Akbar’s company LagerData, that are working on ways to allow for seamless, real-time, and continuous firmware testing. This nut will be cracked soon and hardware-in-the-loop regression testing will be the norm for firmware. In a few years we’ll look back at the dearth of testing we did on firmware prior to 2020 and marvel that we shipped any products at all! 

 

Trend #3 – Getting Development Data out of Silos

A LOT of data gets generated during firmware development and debugging, but most of this data is not stored anywhere. And if it is stored, it is siloed. Consider, as an example, the oscilloscope, the late night friend of many a firmware developer. An oscilloscope generates a lot of data about the performance of the device and additionally captures subtle information about the power supply, fidelity of the data, microprocessor clock operations, among others. But oscilloscope data is often just ephemeral. It is not stored and looked at later. The same is true of data that is displayed in a JTAG debugging console. 

If all this development data could be tagged, sorted, sensibly combined, and stored for future access it could short circuit many of the inefficiencies involved in development. Critical performance questions may be able to be answered without having to set-up and perform a retest. Additionally unsilo-ing this data and making it accessible would pave new and interesting avenues for data scientists and QA engineers to get involved earlier in the product development process.

A massive amount of data gets generated during the development process, but most of this data is either not stored at all, or is stored but is stuck inside a silo. Future IoT development toolchains will bring all of these traditionally siloed data streams together. Photo courtesy of LagerData.

Wrap-Up

Firmware development will continue to grow, but if we can shed the weights of some of the old-school ways of doing things the field will grow faster. If you’d like to learn more on this subject you can watch Akbar’s full Ioterra PDC talk here. Or visit his company, Lager Data, to see how they are creating a powerful development toolkit for the modern, remote firmware developer. 

 

Author: Daniel Price 

Daniel is the cofounder and CEO of Ioterra. An industry expert in Internet of Things, Daniel is deeply interested in new technologies enabling (and being enabled by) the connectivity revolution. Daniel has given a TEDx talk as well as addresses at numerous regional and national conferences about digital transformation from connected intelligent systems. Daniel attended Oxford University on a Rhodes scholarship where he earned an MBA and an engineering master’s degree. For undergraduate Daniel attended UC Berkeley where he graduated summa cum laude with two engineering degrees and a physics minor.

2 Tools That Simplify Product Development

Try Ioterra’s Get Services Bids Tool

Need to bring on expert services partners? Find, communicate with, and get competitive bids from Service Providers

Try Ioterra’s IoT Project Cost Estimator Tool

Get the high level cost and timeline estimates for custom engineering an IoT hardware product