The online information environment is constantly growing in size and complexity and shows no signs of slowing down. This increased scale and speed poses unique challenges for an company looking to leverage such datasets for specific objectives. For counterintelligence efforts, the challenges posed by this dynamism are compounded by the need to have a supremely resilient and secure system to protect against direct malicious attacks. Omelas has found that using a modern serverless architecture provides the best solution in optimizing security, scalability, and resiliency while also limiting costs.
Serverless architecture is a relatively new service offered by cloud computing firms, such as Amazon Web Services, that allows small pieces of code to be run with various time or event triggers without having to maintain or protect an individual server. Each task is run independently by the cloud provider using the most up-to-date hardware and security practices. The ephemeral nature of serverless provides another layer of security. Each task only runs for a few minutes at most and has access to a small part of the system. Any security breach in data collection or processing is inherently limited in scope and duration by the implicit design of the system.
By its very nature, serverless also reduces the number of fixed targets and points of failure. The lack of persistent servers drastically reduces the number of points of failure of the broader architecture. There are no central servers that can be targeted and even if some individual processes are corrupted or attacked, others will take their place automatically from different servers, potentially even in other locations. This reduction in targets is not a complete reduction to zero for Omelas as we maintain a persistent set of databases to ensure stable and accurate data over time. Serverless does drastically limit the potential vulnerabilities by eliminating the need for persistent servers to run the majority of the data collection and analysis.
The benefits to a serverless architecture extend to practical business concerns such as scalability and cost. The fluctuating and unpredictive nature of the volume of publicly available information generated online fits well into the scalable nature of a serverless architecture. The amount of content ebbs and flows due to expected effects such as time of day as well as external events that trigger increased activity. A serverless architecture allows Omelas to have the confidence that the system will hold up and continue to process data with the same speed regardless of how much data is passing through the system. A more traditional system with fixed servers would require a higher overhead to launch more servers or would face a slowdown in data processing. Serverless is able to scale effortlessly with more tasks running concurrently. Costs scale proportionally with the amount of content processed with no additional overhead. A surge in information leads to higher costs that reduce as soon as the surge wanes. Tying costs and usage tightly together allows for a highly flexible platform that limits overhead costs while allowing for large surges in processed data. More complex systems with multiple servers could provide a similar flexibility but with a small team and a highly volatile stream of data, a serverless architecture allows Omelas to have simple yet flexible and cost-effective data collection system.
A serverless architecture with small pieces of code running only when needed provides a strong platform for consuming and analyzing the complex and changing social media environment. The distributed and resilient nature of serverless code allows Omelas to offer a stable and secure platform to assess the information environment. Serverless architecture also minimizes risks ranging from security threats to surges in data to potential cost overruns. Working multiple serverless aspects into our software has made it stronger, more resilient and more timely at identifying emerging trends in the jihadi social media environment.
