Pssst, Cyveillance, I Saw What You Did There

Sunday may be a day of rest for most people but today it's a day of paranoia for me. Maybe not paranoia, cybersecurity is just my third or fourth favorite topic after vice-gripping Bernanke's balls and pulling out the gold plated speculum to see what Goldman Sachs is up to. Speaking of Goldman...

I'm reluctant to even point out that I saw what they did there but fuck it, what else would make me write about this?

Do a quick Google search for Cyveillance and you'll see they're basically an "Internet intelligence" company, which is of course a blanket term for brand monitoring and, in some cases, information farming. Much to my dismay, Cyveillance appears to lean heavily towards that "information farming" side.

Well who cares? (see: Who Is Cyveillance And Why Should You Care?)

First of all, if you're going to watch the watchers, you've sort of got to figure out what gets their dick hard - in the case of Cyveillance (at least according to what I've seen), it's obviously Goldman Sachs. (with a yearly price tag of about $30,000, I could see why the GS rats might find this service useful, as if setting up a Google alert is below them somehow)

And what's with this dirty behavior?

Has your web site been visited by Cyveillance recently? It's quite possible, but you probably wouldn't know it. Cyveillance crawls the net spying on web sites. If you say something they don't like about one of their clients, they'll tattle on you.

Cyveillance uses a couple of dirty tricks when they crawl the web. First, they ignore the robot exclusion protocol. This standard allows you to specify portions of a web site that are off limits to robots and other automatic agents. Cyveillance fails to honor the exclusions you may have declared for your web site. They crawl places that 'bots are not supposed to go, in spite of your explicit instructions not to do so.

This can be a problem for web sites that present deep, dynamic content. For example, I have a spam robot trap on my web site. When a 'bot crawling for email addresses to spam hits that page, the trap is sprung. If the 'bot moves beyond that page, it ends up in a never-ending maze of bogus, generated email addresses. The trap keeps the 'bot tied up, and it fills its database with bogus data.

I don't want to trap well behaved 'bots, such as those used by Google to spider web pages. Therefore, I post an exclusion for this area. This protects the well-behaved 'bot from garbage data, and it protects my website from unnecessary load.

Cyveillance ignores these instructions. Their 'bot gets caught in the trap, crawling places I'm specifically trying to keep 'bots away from.

Another problem with the way Cyveillance crawls is that they provide fradulent header information in the HTTP request. Rather than admitting they are a spy 'bot, they pretend they are a web surfer running Microsoft Internet Explorer.

Personally I don't care, I run a fringe financial blog, I hope whomever is getting the "intelligence" reports is getting a laugh out of this, I saw what you did there but I don't care enough to dig too far or block an entire string of IPs. If Cyveillance is, as suspected, information farming for Goldman Sachs then I would hate for Goldman not to get their money's worth.

I like the sound of "misbehaving spiders" now please back off of my figurative nuts.

But they sure do love the GoldmanSachs666 and no one had to pay me $30,000 a year to figure that out (although these people might be on to something...).

Cyveillance's parent is even more frightening, a break-off of British intelligence who acquired Cyveillance earlier this year.


A Pentagon office that claims to monitor terrorist threats to U.S. military bases in North America -- and was once reprimanded by the U.S. Congress for spying on antiwar activists -- has just awarded a multi-million dollar contract to a company that employs one of Donald Rumsfeld’s former aides. That aide, Stephen Cambone, helped create the very office that issued the contract.

On January 7, QinetiQ (pronounced “kinetic”) North America (QNA), a major British-owned defense and intelligence contractor based in McLean, Virginia, announced that its Mission Solutions Group, formerly Analex Corporation, had just signed a five-year, $30 million contract to provide a range of unspecified “security services” to the Pentagon’s Counter-Intelligence Field Activity office, known as CIFA.

According to Pentagon briefing documents, CIFA’s Directorate of Field Activities "assists in preserving the most critical defense assets, disrupting adversaries and helping control the intelligence domain.” Another CIFA directorate, the Counterintelligence and Law Enforcement Center, "identifies and assesses threats" to military personnel, operations and infrastructure from "insider threats, foreign intelligence services, terrorists, and other clandestine or covert entities," according to the Pentagon. A third CIFA directorate, Behavioral Sciences, has provided a "team of renowned forensic psychologists [who] are engaged in risk assessments of the Guantanamo Bay detainees."

The new CIFA contract with QinetiQ expands work that Analex has provided CIFA and its various directorates since 2003. Under its first contract, according to the QinetiQ website, Analex staffers were sifting through information “from traditional to non-traditional providers, ranging from unclassified through top secret classification using sophisticated information technologies and systems specifically designed by CIFA analysts.”

The CIFA contract was awarded just two months after QinetiQ hired Stephen Cambone, the former undersecretary of defense for intelligence and a longtime Rumsfeld aide, as its vice president for strategy. Cambone is the most senior of a savvy group of former high-ranking Pentagon and intelligence officials hired by QinetiQ to manage its expansion in the U.S. market. (See boxes.)

While he was at the Pentagon, Cambone oversaw CIFA and was deeply involved in the Pentagon’s most controversial intelligence programs. It was Cambone, for example, who reportedly issued orders to Major General Geoffrey Miller to soften up Iraqi prisoners for intelligence interrogators in Abu Ghraib in 2003. With Rumsfeld, he also set up a special unit within the Pentagon that alienated the CIA and the State Department by running its own covert actions without seeking input from other agencies.

Oooooh frightening, I better stop talking about it as we're now in the "I have no idea what I'm talking about" territory.

Legal disclaimer: I did actually see what you did there. And I will tell. (if no one hears from me for awhile, someone tweet my ass or something)

Jr Deputy Accountant

Some say he’s half man half fish, others say he’s more of a seventy/thirty split. Either way he’s a fishy bastard.


Hi, thanks for the article. I keep getting hits from these guys and it creeps me out. I decided to google them and came onto your page. I'm not doing anything illegal so I should be ok. But still, what the heck is the internet turning into with companies like these invading our websites.

Your Roomba repair skills must be a threat to national security, guy!

They've mostly been leaving me alone lately but if you see any hits from Psinet in your logs, it's probably the same stalkery crap.

W.C. Varones said...


Did you wipe it off?