DARPA government research agency publishes catalog of open source projects

No readers like this yet.
An open card catalog
Image by:

Opensource.com

The Defense Advance Reseach Project Agency (DARPA) is one of the government-sponsored research agencies that most boldly explores the future of science and technology. Given that many of its research projects have military applications, it has been traditional for the agency to be secretive about them. In recent years, however, DARPA has been embracing the benefits of open source, particularly for promoting rapid innovation. Last week, the agency opened to the public a new portal featuring a catalog all its open source projects.

Past breakthrough achievements of DARPA include:

Some have described the culture of the agency as one that celebrates 'mad scientists' going around building and creating technologies to bring the future about. With an annual budget of $2.8 billion, DARPA drives a good portion of the advanced research that happens at universities and corporations in the US.

The new catalog features not only open source software that has been sponsored by the agency, but also open access publications of its funded research. All content that is sensitive to military applications was removed.

Most of the projects in the new catalog are related to the XDATA initiative, that is developing open source infrastructure for supporting big data.

Among the salient projects in the catalog we can find:

  • Vega: visualization grammar
  • Lyra: interactive visualization environment
  • LineUp: scalable rankings visualization  
  • Blaze: the next generation of NumPy
  • Tangelo: HTML5 web server architecture platform
  • OODT: Apache project for distributed resources
  • WINGS: workflow for designing scientific experments
  • OZONE: widget framework (that I covered here)
  • Spark: Apache project engine for large-scale data processing
  • VMR: Visual Media Reasoning

Explaining the motivation for this move, the agency's press release states:

Many DoD (Department of Defense) and government research efforts and software procurements contain publicly releasable elements, including open source software.

The nature of open source software lends itself to collaboration where communities of developers augment initial products, build on each other's expertise, enable transparency for performance evaluation, and identify software vulnerabilities.

DARPA has an open source strategy for areas of work including big data to help increase the impact of government investments in building a flexible technology base.  

The press release also elaborates on the importance of building collaborative communties around these open source projects:

"Making our open source catalog available increases the number of experts who can help quickly develop relevant software for the government," said Chris White, DARPA program manager.

"Our hope is that the computer science community will test and evaluate elements of our software and afterward adopt them as either standalone offerings or as components of their products."

This new catalog of open source software is a great step towards bringing the benefits of rapid innovation through open source to government, industry, and academia.


Tags
Government
Science
Luis Ibanez
User profile image.
Luis Ibáñez works as Senior Software Engineer at Google Inc in Chicago.
More about me

2 Comments

Avatar
David A. Wheeler | February 10, 2014
No readers like this yet.

I think it’s great that the government is releasing government-funded research software as OSS; this should help spur research.

There are even formally-published academic articles that note the key advantages of releasing software as OSS in research. Below are two examples.

“… The release of Otter at CADE-9 in 1988 was a turning point in the history of automated reasoning. Never before had the computer science community seen a theorem prover of such awesome power… perhaps Otter’s greatest impact was due to Bill’s generous and far-looking decision to make its source code publicly available. It is impossible to describe completely a reasoning program in research papers. There is always some amount of knowledge, often a surprising amount, that is written only in the code, and therefore remains hidden, if the code is not public or is too hard to read. Bill’s code was admirably readable and well organized. Other researchers, including those whose systems eventually overtook Otter in speed or in variety of inference rules, also learnt from Bill’s code data structures, algorithms, and indexing schemes, which are fundamental for implementing theorem provers…” - “Automated Reasoning and Mathematics: Essays in Memory of William W. McCune”, Lecture Notes in Artificial Intelligence 7788, preface, http://link.springer.com/book/10.1007/978-3-642-36675-8

“From the publications alone, without access to the source code, various details were still unclear… what we did not realize, and which hardly could be deduced from the literature, was [an optimization] employed in GRASP and CHAFF [was critically important]… Only [when CHAFF’s source code became available did] our unfortunate design decision became clear… The lesson learned is, that important details are often omitted in publications and can only be extracted from source code. It can be argued, that making source code … available is as important to the advancement of the field as publication.” - The paper ”The Evolution from LIMMAT to NANOSAT” by Armin Biere (April 2004), http://fmv.jku.at/papers/Biere-ETH-TR-444-2004.pdf

You can see more context about this in http://www.dwheeler.com/blog/2014/02/07/#william-mccune-source-code if you're curious.

Avatar
Luis Ibanez | February 10, 2014
No readers like this yet.

David,

Thanks for bringing up the connection between open source, and reproducibility and open science. They truly build upon each other to accelerate the pace of discovery and innovation.

The Open Science movement is gaining momentum:
https://opensource.com/life/13/12/tedx-talk-saving-science-open-or-perish

and more recognition is being given to the need for including open source implementations as part of scientific publications. Still a lot remains to be done in that front. So, reminders like yours are very helpful to make the case for why openness should be a requirement.

Related Content

Creative Commons LicenseThis work is licensed under a Creative Commons Attribution-Share Alike 3.0 Unported License.