“How can we connect the dots when we don’t have all the dots?”

Last Updated on Friday, 7 November 2008 05:53 Written by admin Friday, 7 November 2008 05:53

The Defense Science Board report that came out 3 days ago continues to make headlines in Government circles. Today’s NextGov.com article underscores the reports findings about the need to include Open Source intelligence in counterterrorism activities.

(t)he board said. “The No. 1 issue in counterterrorism is that we are information-limited,” the report stated. “Many nostrums for improving intelligence in support of counterterrorism focus on ‘connecting the dots’ on the presumption that we have all the dots. We do not, nor are we sufficiently astute and aggressive enough in collecting them.”

The report, “Defense Imperatives for New Administration”, released on Nov. 4, said combating terrorism requires putting domestic intelligence collection on par with foreign intelligence. The creation of the Office of the Director of National Intelligence, which placed all intelligence agencies under an umbrella organization, was supposed to achieve this parity. But the science board said, “successive directors of national intelligence have been slow to embrace domestic intelligence, and that must be remedied.”

The report slammed the agencies for their disdain of open source intelligence, saying that since the Sept. 11 attacks “every commission and every observer and critic of the intelligence community” has touted the value of such information channels.

“Much of what we know about terrorist groups comes from open sources,” the board said. “Much of what we do not know and need to know is to be found in open sources.” But the intelligence community “retains a propensity to undervalue and shortchange” them, the report said.

Matt Burton: “Set Your Analysts Free.”

Last Updated on Friday, 3 October 2008 09:57 Written by admin Friday, 3 October 2008 09:57

Great post by Matthew Burton on the value of Open Source. He underscores my own inspiration for Project Grey Goose when he writes about the phenomenal but ultimately tragic Jim Gray rescue effort last year:

Freedom: The Jim Gray Rescue Effort

The Web is fast and free: fast, in the sense that you can make things happen overnight; and free, in the sense that you don’t need the boss’s permission to implement new ideas. This was no more apparent than in the case of Jim Gray.

Jim Gray was a database pioneer based in San Francisco. He disappeared while sailing in the Bay area last January. He was never found.

But his friends, many of whom were Silicon Valley magnates, did their best to try. One of these friends retasked DigitalGlobe imagery satellites to shoot fresh photos over 3,500 of sea. The result was 1400 gigantic images. So coders automatically split the images up into 560,000 smaller ones that could be easily reviewed; someone else contributed a computer program that automatically sharpened all of the images.

Now that they had over a half million pictures, they needed a way to solicit volunteers to look at them. Amazon.com already had a ready-made solution, so the images were posted on a special page that let virtual volunteers cull through these images in search of Gray’s sailboat. 12,000 volunteers reviewed all 560,000 images three times each.

All of that happened in three days. They didn’t find Gray’s sailboat. But it was the largest search party in history, on top of being simply amazing.

The Web’s speed and freedom made it possible. Had this job been given to an organization with a closed network, this never would have happened.

What about your organization’s intranet? Could your IT team split up those images into easy-to-consume portions? Could they write a script that automatically sharpened all 560,000 images? Could they create a tool that lets the rest of your workforce review the images from their own desks? Could your servers handle the load? And could all of that be accomplished in three days? In order to do that, you need to have the technical talent on hand.

Once you have the talent, those people need permission to work freely and create solutions on the fly. Do they have that permission? Or would your org have to submit justification statements, get contract managers to approve personnel reassignments, and run each line of code past the infosec people before finally getting something live?

On the Web, if you have an idea, you just do it. This is a philosophy and capability the Intelligence Community (and any large, data-driven organization) should keep in mind if it wants to fully exploit its data. Analysts will always have too much information to process and too little time in which to do it (and focusing the “open source intelligence” conversation on culling Web content contributes to that problem). To have any hope, they need an IT and bureaucratic infrastructure that will accommodate improvised solutions.

Again: if you want to make the most of open source, get an open network that lets analysts contribute their own material. Put all of your analysts onto one large network instead of letting them work on lots of small ones. Hire some developers and familiarize them with the analysts’ information problems. Then, set them free and let them experiment.

Classifying OSINT Products is a “Grey” Area

Last Updated on Monday, 22 September 2008 05:08 Written by admin Thursday, 18 September 2008 07:15

There’ s been a lot of discussion recently about the need to restrict access to output produced by Open Source analysis.

DangerRoom highlights CIA Director Hayden’s interest in classifying Open Source intelligence. Secrecy News‘ Steven Aftergood blogs about “the most coherent argument for open source secrecy I have heard” – that of Jennifer Sims, director of Intelligence Studies at Georgetown University who says that if the result of OS analysis provides a “decision advantage”, then access to that analysis requires classification. However, Aftergood adds:

“Only a small minority of OSC analytical products fits that description,” he told UPI. “The overwhelming bulk have no operational relevance. They are at best contextual,” he added, comparing them to the research backgrounders prepared by the Congressional Research Service. ”They are not inputs into strategic decision-making.” 

I’ve thought a lot about how open we can be with our own grassroots effort Project Grey Goose. And the answer is – not very. Russian visitors to this blog have gone up 500% since I first announced the launch of Grey Goose on August 22, and it’s still climbing. That should serve to remind all of us in this field that Open Source is a double-edged sword. 

I’m also worried about a gleam that I see developing in the virtual eyes of the leadership of the Intelligence Community regarding OSINT, particularly when it was treated so shabbily just a few years ago. If OSINT analysts are getting results and making a difference, it’s due in large part to the freedom of developing our own sources and methods, workflows and data sources, and even inventing new technology (the “other” Open Source) as we see fit. I agree that reports which provide “decision advantage” should be restricted. I only ask that the IC recognize and respect “how” these reports got made in the first place, and not do anything to harm that “open” approach.