FileMaker is an excellent rapid application development tool. We can prototype things in FileMaker in a quarter of the time it might take in another environment. Portage Bay Solutions
FileMaker is an excellent rapid application development tool. We can prototype things in FileMaker in a quarter of the time it might take in another environment.
Automated Identification System is a technology used in marine transportation to track the movement of ships and prevent collisions. AIS uses a combination of GPS and VHF radio signals to transmit information about a ship’s identity, speed and location to other ships and to shore-based monitoring stations. As a result, AIS makes marine transportation safer and more efficient by providing real-time data about ship movements. In principle, it is similar to the transponder system used by the aviation industry to keep track of the position of planes in the air.
In the same way that aviators and flight enthusiasts use FlightAware to track aircraft, AIS is available via the popular ShipFinder app on a smartphone or on their website. Visiting Marinetraffic.com is another option. Both services allow access, with some restrictions, to vessel positions of marine traffic around the globe.
Our client for this project manages a network of ground-based stations which receives the AIS messages from all marine traffic in Washington’s Puget Sound. They use an off-the-shelf solution to view the AIS data in their region, displayed as a map overlay. While this allows our client to access vessel position info, they became interested in a solution branded for their own organization which gave them better control of the data, and better reporting capabilities as well.
Typically, Claris FileMaker may not be the first development tool selected for such an application. It lacks the native capabilities to read the UDP datastream of the AIS messages output by the receiving station, and the amount of data to be dealt with is immense. Yet FileMaker is an excellent rapid application development tool. We can employ rapid prototyping in FileMaker in a quarter of the time it might take in another environment.
Since we cannot natively read a UDP stream in FileMaker, we turned to building a custom Node.js application instead. Our developer, Jacob Carr, combined the Net module, FS module and an open source community module, used for decoding AIS messages. This service is used to listen to the UDP data stream and output the data into JSON objects in a series of .ndjson files.
Importing the raw data into FileMaker exceeded FileMaker’s performance limits for this type of import. We tweaked the micro-service to throttle the data stream in such a way that the server side script in FileMaker Server could efficiently keep up with the data.
With the raw data now in FileMaker tables, it was a straightforward Google Maps API implementation to display the vessel locations on a map of Puget Sound in a WebViewer object.
If you are interested, Portage Bay has discussed Google Maps integrations in past blogs, which you can access here:
Our prototype was now in place and we were able to show our client their data on the map. Simple FileMaker finds allow the user to see all vessels going in a particular direction, above a certain speed, or destined for a particular port. In addition, historical views can be seen – all vessels in a particular area at a previous point in time, or the location of a particular vessel in Puget Sound at any time in the past.
We decided to make one final Google Maps tweak to further refine the picture of vessel data we were displaying. The Maps API allows you to define custom icons for your map pins, so we created 24 arrow shaped icons pointing at compass headings from 0 to 360 degrees, in 15-degree increments. Assigning the icons based on the vessel’s actual heading from their AIS transmission allows someone to see each vessel’s current heading at a glance.
FileMaker allowed us to rapidly create a working application for viewing AIS data on a Google Map interface, and to provide the client with basic tools for customizing views of that data.
Our client is now experimenting with the prototype and deciding where they want to go next. It may stay in FileMaker, or we may combine it with big data tools for faster reporting on larger and larger data sets. The solution currently receives an average of around 114,000 messages per hour and over time the historical data will get more difficult to report on efficiently. It is also possible that we will use the FileMaker prototype as a model for a future JavaScript-based web application. Time will tell!
We designed a partially custom built micro-service utilizing Node.js. Primarily, we are using standard methods available via the default LTS, with the main goal being to listen to and transpose data received via a UDP connection. We leveraged the “Net” module which provides an asynchronous network API to create a stream using sockets to listen in on the UDP transmission. The stream can then be captured as binary data and converted into its raw structure as AIS or NEMA encoded messages.
Once we established a consistent connection and were able to log the results of the data stream, we were faced with decoding the NEMA messages. We turned to a community module for AIS Decoding which transposed the encoded data streams into individual JSON objects. Finally we output this data to NDJSON files using the “FS” module.
Now that the data collected was in a much more manageable state, we could push the new data structure into an existing database. Since the datasets received from the UDP transmission arrive in a constant stream of live data, creating individual records for every message proved nearly impossible for FileMaker to manage alone without significant performance issues.
We needed to go one step further by creating a throttle in the node file to slow the existing stream and limit the record creation to about ten records per minute. We have since been able to sustain a healthy transaction between the node-created messages and the records in FileMaker without losing any data.
Do you wonder whether something is impossible to achieve within FileMaker? It’s probably not. Claris is trusted by over 50,000 of the world’s greatest companies, with over one million people using the platform to transform their workplaces, and Portage Bay has been assisting in that work for over thirty years. Send us your thoughts, so we can begin solving the puzzle together.
John Newhoff’s business background and years of database design, computer, and network configuration, and troubleshooting experience allow him to see beyond the immediate problem to the long-term solution. John is the business manager of Portage Bay and our lead 4th Dimension developer, creating sophisticated cross-platform solutions.
Your email address will not be published. Required fields are marked *
Comment *
Name
Email
Website
Save my name, email, and website in this browser for the next time I comment.
Post Comment
