This entry was posted on 1/27/2006 9:00 AM and is filed under Cybercrime.
An interesting, albeit questionable, survey was released earlier this month that has drawn a lot of media attention as well as other interested groups. Generally speaking, the available public references cite the report as authoritative. On the other hand, private discussions amongst information security professionals paint a different picture. It is interesting to note that the official report, first mentioned on the FBI's website on January 18, 2006 does not provide a link to the report whereas the Department of Homeland Security's Daily Open Source Infrastructure Report dated January 13, 2006 in paragraph 41 refers to the report and provides a link to the source which is no longer valid. Links to a few of the reports include:
There are many more to be found by using your favorite search engine with the value "2005 FBI Computer Crime Survey Report" or "2005 FBI Computer Crime Survey Report.pdf". The difference in results using both terms is quite surprising. Ignore any of the links referring to the "CSI/FBI" surveys as they are quite different.
1/27/2006 9:50 AM
Craig A Schiller CISSP wrote:
The news report said:
"Use of antivirus, antispyware, firewalls and antispam software is almost universal among those who responded. But the software apparently did little to stop malicious insiders."
Isn't that special! And I suppose that new compiler I bought won't correct my spelling mistakes either. Antivirus, antispyware, firewalls, and antispam are primarily intended for the external threat. The finding is irrelevant.
If they had wanted to compare apples to apples, then they would have talked about file access controls, role based access, segregation of duty (SOD), logging, configuration of the log file, internal use of IDS/IPS, change management, change monitoring of critical systems, administration of user access with regard to new users, role changes, and terminations, reconciliations of access, roles, and SOD, background checks, etc.
But wait! Stop the presses! The reporter didn't quote the report although the article sure makes it look like he did. Here is the actual finding in the report.
"Although the usage of antivirus, antispyware, firewalls, and antispam software is almost universal among the survey respondents, many computer security threats came from within the organizations."
Doesn't quite convey the same message as the reporter's version does it?
I noticed on the front page of the actual report that this report was not produced with CSI as the ones in the past were. The list of major contributors does not include a single information security officer. It includes two criminals, two FBI agents, two consultants, and a host of our friends in the academic community. Neither of the consultants spent any time as an information security officer.
Is anyone else bothered by the use of quotes from two criminals on the front cover?
"The 2005 FBI Computer Crime Survey should serve as a wake up call to every company in America."
Frank Abagnale . Author and subject of 'Catch Me if You Can' . Abagnale and Associates
"This computer security survey eclipses any other that I have ever seen. After reading it, everyone should realize the importance of establishing a proactive information security program."
Kevin Mitnick . Author, Public Speaker, Consultant, and Former Computer Hacker . Mitnick Security Consulting
The FBI has the entire universe of security professionals to choose from and they pick two criminals for the front cover! Maybe the couldn't get good quotes from anyone that wasn't self-serving.
The survey only covered four states (New York, Texas, Iowa, and Nebraska) and was conducted by FBI offices in those states. Reply to this
1/27/2006 9:53 AM
Craig A Schiller CISSP wrote:
Ira Winkler's assessment is an excellent read. However, I do disagree with the position that, in general, these types of surveys are not useful. Unfortunately this one has too many mistakes and inappropriate conclusions to be useful.
Here's an example.
"Interestingly, having more security measures did not mean a reduction in attacks. In fact there was a significantly positive correlation between the number of security measures employed and the number of Denial of Service (DoS) attacks."
In what meaningful way does the number of security measures affect the number of attacks? Two hypotheses might explain the correlation - 1) bigger companies are subject to more attacks than smaller companies or 2) Companies with more security measures detect more attacks than companies with fewer security measures.
A far more useful correlation would be to see if organizations with more security measures detected more attack attempts than they did when they had fewer measures. That would be a trending metric which this survey doesn't address.
The way the reporter presented the quote seems to suggest that companies should deploy fewer security measures so they will be attacked less. Nonsense! The sentences following the quote attempted (poorly) to clarify the meaning of the finding - that companies with more security measures were likely to detect attacks better than those with fewer measures.
Here's another interesting observation - of the respondants, only 1.8% were information security officers! That sounds really wrong until you look at the companies in the survey. 20% of the companies had fewer than 10 employees. > 70% had fewer than 100 employees. This might explain why 13% said they had no security incidents. In smaller companies no one has time to keep track of security incidents. In fact, until a company is big enough to have a dedicated security officer, its unlikely they will know anything about their security incidents unless the security incident was big.
Once, I worked as the first security officer for a multi-billion dollar international corporation. In the cover letter to their information security policy it actually said, "We've never had a security incident, but security is probably a good thing so we ought to do it." In my first year with them I documented over 120 significant security incidents. It is more likely that every one of the companies in the survey had many more security incidents, as defined in the survey, than they reported but they suffer from "a tree falls in the forest that no one hears" syndrome. Reply to this
1/27/2006 10:00 AM
Craig A Schiller CISSP wrote:
I believe that the FBI was irresponsible in releasing the survey without a critical review. From the quotes in the survey I get the feeling that the academics were asked if they could contribute comments much like you would put on the back cover of a book to market it. As Ira said, the Academic community quotes appear to have been added to give the survey an air of authority, as if they had participated and quided the survey. The survey suffers from bad survey technique as well as poor security interpretation of the results. The news report compounds this with bad journalism.
The report said: "The purpose of this survey is to gain an accurate understanding of what computer security incidents are being experienced by the full spectrum of sizes and types of organizations within the United States."
But the first finding says:
"There are a variety of computer security technologies that organizations are increasingly investing in to combat the relentless, evolving, sophisticated threats, both internal and external. Despite these efforts, well over 5,000 computer security incidents were reported with 87% of respondents experiencing some type of incident."
If that was the purpose of the report, why do the findings draw conclusions about about the effectiveness of security expenditures? Note, there are no questions about the effectiveness of security measures, there is only the finding that despite increased security spending, there were security incidents. The finding should have started with "well over 5,000 computer security incidents ..." and drawn no inferences or references to the security investment.
How about this finding:
"An overwhelming 91% of organizations that reported computer security incidents to law enforcement were satisfied with the response of law enforcement."
Go figure. No possible bias there from companies willing to cooperate with the FBI by participating in their survey. You can't expect to get good data on a question like this if the answer is collected by the organization in question. This would have to be part of an independent survey from a non-partison third party to be meaningful. Reply to this
