Final Report for Google Summer of Code 2023
Project: Support basic SensorThings API (STA) for istSOS
Organization: Osgeo (Open Source Geospatial Foundation)
Abstract
During the Google Summer of Code 2023, I worked on the project “istsos-miu”1 part of the OSGeo2 Foundation. The goal of the project was to create an implementation of the Sensor Things API3 standard for the istsos project. The Sensor Things API is a standard created by the Open Geospatial Consortium (OGC) for the Internet of Things (IoT). The standard defines an open and unified way to interconnect IoT devices, data, and applications over the Web. The standard is based on the RESTful architecture style and uses the HTTP protocol for data transfer.
The state of the art before GSoC
Before GSoC there was an implementation of the Sensor Things API standard written in Java, but we wanted to create a new implementation in Python to be able to integrate it with the istsos project. So the project was built from scratch.
What I have done
Throughout the Google Summer of Code 2023, my primary objective was to enhance the functionality of the istsos project by aligning it with the standards set forth by the Sensor Things API3. I successfully achieved this goal by contributing to various components of the project, each of which played a vital role in its development and improved usability. My contributions can be categorized as follows:
STA Language Parser and Lexer
One of the cornerstones of my work was the design and development of a dedicated parser and lexer for the Sensor Things API (STA) language, which I named “sta-parser”4. This parser and lexer duo served as a fundamental interface, enabling users to interact with the istsos server through the STA language – a query language specifically tailored to the Sensor Things API. The parser and lexer, packaged as a standalone solution, hold potential for use in other projects beyond istsos as well. While the parser is continuously evolving, it currently boasts support for all the essential features of the STA language. This achievement opens the door to further improvements, making the STA language more accessible and intuitive for users.
API Service Creation
A substantial portion of my efforts was dedicated to the creation of a new API service within the istsos project. This service is pivotal in enabling users to seamlessly interact with the istsos server by utilizing the capabilities of the Sensor Things API. The newly developed API service facilitates a wide array of operations, including:
- STA Language Queries: Users can now execute queries using the STA language, providing a standardized way to request data from the istsos server.
- Data Insertion: The API service empowers users to insert new data into the system, streamlining the process of populating the database with essential IoT data.
- Data Updates and Deletions: According to the STA standard, users can effortlessly update and delete data entries using the API service, ensuring data accuracy and relevancy.
This comprehensive API service acts as a bridge between users and the istsos project, making it easier than ever to harness the power of the Sensor Things API for effective IoT data management.
The current state of the project
The goal of achiving an implementation of the Sensor Things API for the istsos project has been achieved. The project is now in a state where it can be used by users to interact with the istsos server using the Sensor Things API. The project is still in development and will continue to be developed in the future.
What is left to do
While significant progress has been made during the Google Summer of Code 2023, certain features from the official Sensor Things API standard were not able to be fully implemented within the given timeframe. These missing features include:
MQTT Extension:
Implementing the MQTT extension for seamless communication between IoT devices and applications.
Data Array Extension:
Incorporating support for data arrays to enable efficient storage and retrieval of multiple data points.
MultiDatastream Extension:
Introducing the MultiDatastream extension to accommodate scenarios involving multiple data streams.
Batch Request Requirement:
Developing the capability to handle batch requests efficiently.
Historical Location Auto-Creation Requirement:
Addressing the auto-creation of historical location information, ensuring accurate historical data representation.
Pagination Requirement:
Implementing data pagination to manage and present large datasets effectively.
Built-in Query Functions Requirement:
Adding built-in query functions to simplify and enhance data querying capabilities.
Data Count Requirement:
Incorporating data count functionality for accurate monitoring of data volume.
Status Code Requirement:
Implementing support for querying data based on status codes, enhancing data analysis possibilities.
Although these features could not be fully realized within the project timeline, they represent potential areas for future development and expansion of the “istsos-miu” project. Their implementation would further enrich the project’s capabilities and align it more closely with the comprehensive Sensor Things API standard.
It’s worth noting that while these features remain outstanding, the current contributions have laid a strong foundation for their potential integration in the future.
What code has been merged
All the code that I have written during the Google Summer of Code 2023 has been merged into the traveltime branch of the istsos project. The code can be found in the following repositories:
The list of pull requests that have been merged can be found here:
What I have learned
Throughout the course of the Google Summer of Code 2023, I have gained a wealth of knowledge and experience that has significantly enriched my skills and understanding of software development, programming languages, and open-source collaboration. The journey of contributing to the “istsos-miu” project and aligning it with the Sensor Things API standard has taught me invaluable lessons in various areas:
Language Design and Parsing
I have learned a lot about language design principles and the intricate process of creating parsers and lexers. Designing the “sta-parser” for the Sensor Things API language required a deep dive into the world of parsing techniques, grammars, and syntax analysis. This experience not only sharpened my coding abilities but also honed my problem-solving skills in tackling complex challenges related to parsing and interpreting domain-specific languages.
RESTful Architecture and API Design
Working on the “istsos-miu” project exposed me to the nuances of RESTful architecture and API design. I gained insights into designing interfaces that follow best practices for exposing functionalities over the web. Crafting the API service to seamlessly integrate with the istsos server using the capabilities of the Sensor Things API taught me the importance of standardized communication and data exchange protocols in the context of IoT.
Collaboration and Open Source Development
Being part of an open-source project under the OSGeo Foundation has been an incredible experience in collaborative software development. Through code reviews, discussions, and interactions with mentors and other contributors, I learned how to effectively collaborate in a remote and diverse environment. The feedback received during the review process has immensely improved the quality of my code and enhanced my ability to communicate technical ideas clearly.
Python Proficiency
While Python was not my primary language before this project, I have grown significantly more proficient in it due to the nature of the tasks I undertook. Developing the parser, lexer, and other components of the project in Python expanded my programming language repertoire and gave me a broader toolkit for future software endeavors.
Project Management and Time Management
The Google Summer of Code experience required effective project management and time management skills. Balancing the goals of the project, meeting milestones, and overcoming unexpected challenges within the given timeframe taught me how to prioritize tasks and manage my efforts efficiently.
Technical Research and Adaptation
Implementing the Sensor Things API standard demanded a thorough understanding of the technical specifications and principles behind it. This required extensive research, learning, and adaptation of concepts related to IoT, RESTful APIs, and data querying. The experience of diving into unfamiliar territory and rapidly grasping new concepts has made me a more adaptable and resourceful developer.
Conclusion
As the Google Summer of Code 2023 comes to a close, I am filled with gratitude for the incredible opportunities provided by Google and the OSGeo organization. This journey of contributing to the “istsos-miu” project, aligned with the Sensor Things API, has been a transformative experience.
Undoubtedly, the project’s scope exceeded the initial proposal, hinting at the potential for a more expansive endeavor. Nevertheless, the project has achieved a level of functionality that aligns with the primary objectives of implementing the Sensor Things API within the istsos project. Users can now seamlessly interact with the istsos server using the standardized interface, paving the way for efficient IoT data management.
I extend a heartfelt thank you to Google and the OSGeo organization for providing me with this invaluable opportunity to contribute to open-source development. My deepest appreciation goes to Massimiliano and Daniele, whose mentorship, feedback, and support have been instrumental throughout this journey. Their dedication and expertise have guided me in overcoming challenges and pushing the project to new heights.
As I reflect on this experience, I recognize the substantial strides made in enhancing my skills and understanding of language design, API development, collaboration, and more. This journey has equipped me with the tools to tackle complex technical challenges and collaborate effectively within the open-source community.
While certain features of the Sensor Things API standard remain on the horizon for future development, I am proud of the foundation we have built. The merged code in the istsos project repositories stands as a testament to the collective effort put into this endeavor.
As I look ahead, I am excited to continue contributing to the open-source community and leveraging the lessons learned from this project to pursue new opportunities.
Thank you once again to Google, the OSGeo organization, Massimiliano, Daniele, and all those who have supported me on this remarkable journey.