100 Gigabytes of sensitive data belonging to a cooperative division of the US Army & NSA was left on a publicly-accessible Amazon web server, according to ZDNet.
The data, which was part of a US Army INSCOM program called “Red Disk,” was found by the director of the infosec firm UpGuard, Chris Vickery, in October. It was stored on an Amazon Web Services server that was unlisted but not password-protected. Vickery notified government officials and the server has now been protected, but its owner is unknown.
Red Disk was an early, unsuccessful cloud-based data program that was developed for the Army to streamline the gathering, processing and analysis of intelligence data. But the program was slow, buggy and crash-prone and it was ultimately abandoned after $93 million was spent in development costs.
Red Disk could draw in vast amounts of intelligence, documents, videos, and audio from several sources, including signals intelligence, radar, wide area aerial surveillance, drones, and audio databases — some fed in directly from the NSA. That raw, mostly unstructured data passed through software called NiFi (formerly NiagraFiles), a since declassified NSA system to support highly scalable and flexible data flows, which directs different kinds of data across multiple computer networks and geographically dispersed sites. That was particularly useful for Red Disk, which relied on obtaining and sending data over wide areas.
The data then was sorted and organized through various filters. The data would be indexed, allowing analysts to carry out metadata tagging, extract geo-temporal information, and run a data provenance process to verify the source and owner of certain data.
All the collected intelligence would be stored in a central repository to be analyzed, correlated, and enriched. An analyst could pull intelligence from the repository based on their security clearance. An analyst would obtain their access from their Pentagon-issued certificate-based credentials, which grants them access only to data they are permitted to see.
Vickery also noted the presence of security keys and other sensitive files belonging to the software’s outsourced development partner, Invertix (now called Altamira Technologies).