At the Middle of my Masters at MIT Media lab, I was starting to learn how to use Thermal Cameras on the thesis project, the problem was that the system was getting more and more complex, while de data visualizations where the usual flat 2D plots. Because the system was a spatial computing platform, meaning it computes information from the 3D physical space, I felt the need to have a tool to plot in 3D, moreover, real-time data from sensors on the space, in 3D.
The First, and most used, sensor was a Panasonic AMG8833 thermal array sensor.
The Following step was to integrate, combined with real-time code update, real-time data streams, like MQTT, a communication protocol vastly used on Internet of Things devices - also topic of my masters thesis.
System Evolution and Modules
Over time, due to the simplicity of the system, I started adding modules, like Maps 3D visualizations, support to OBJ and FBX 3d model files, support to other communication protocols beyond MQTT, like Bluetooth.
The very first IDE was inspired on Processing, a project also started at the MIT media lab. The Application was developed using Electron, a JavaScript framework to build cross-platform desktop apps using Web Technologies. The 3D environment is based on Three.js - a WebGL framework that abstracts most of the low-level applications. The app also had, like Processing, an integrated code editor and a buttons to ‘run’ the code.
The idea was simple, have code being uploaded whenever there is a change on the text. simple and just that already save quite a good time of 3D viewer development. However on later versions I changed the integrated UI to external code editors, like Visual Studio code or Sublime. The reason for that was simple, developers are picky when it comes to their favorite IDEs, and I feel hard to use external IDEs on Processing and Arduino. Therefore I implemented a project protocol and file watcher to keep track of changes in external files.