Monday, July 23, 2007

Thoughts on Security Conferences versus Practical Knowledge

Over on Don Parker posted an article on Security conferences versus practical knowledge.

Overall I see his point that the talks given at the average security conference actually gives little to the average participant to bring home to put into effect into his/her network. He asserts that the training given at conference (usually 2+ days before the talks) is top notch but the speakers fall short. He also says that a security conference focusing on "practical knowledge" would be far better.

From the article:
"Today's computer security conferences no longer offer relevant or practical knowledge to the attendee. Be honest now, when was the last computer security conference that you went to where you came away from with several ideas to implement immediately onto your networks? I would wager none. "
"What my not making the cut sank home for me though was that there are precious little practical talks going on today at computer security conferences."

Some thoughts on those quotes:
We have done this to ourselves by demanding that we hear talks on the latest research and 0-day, brand new exploit attack vector, uber l33t hack tool, etc when we go to these security conferences. At some point we moved away from talks on practical widespread attack vectors on our network to teeny tiny attack vectors because all the "practical talks" have been given already and why do people want to pay tons of money to hear someone talk about research or information that everyone already knows?

When was the last time i got something useful from a security conference? The last con i went to was shmoocon 07 (My posts on about it 1, 2, & 3)and while i wasnt able to go back to work, sit down at the domain admin MMC or router console and make changes that secured my network i still got alot out of the con. You can read my day by day if you want, but i'll assert that being able go back and make a change or implement something new on your network after a security con attendance is a poor metric to judge a conference selection of speakers or the value of the conference. Talks i did get alot out of were:

Avi Rubin's keynote talk on vulnerability disclosure. Do i do this every day, no. But great information to know when i have a enough fu to worry about doing disclosures.

Matt Fisher, Cygnus, and PresMike's talk on Web Application Incident Preparation. Again, i dont run a web server but if i did i would have gone back and looked at what we had in place to deal with incidents that could occur thru my web app.

I missed Richard Bejtlich's talk but i'll wager it was worth listening to :-)

Chris Paget's talk on WPAD, if we were using it, would have been a talk i would have had to sit down at the keyboard and do some fixing on.

There was more, i wont list them all, hell even the guys talking about guns was worthwhile but not something i could have used at work.

Link to the speakers

so what's my point???!!! first another quote...

"It is not everybody who can attend today's cutting edge security conferences and actually walk away having learned something. What is it that you are going to get out of it, and just how will it benefit our network? If the answers aren't there, you're not going. Practical knowledge is where it is at."

My point is that i think people (anyone if they have some brain cells and interest) do get things out of conferences even if they cant directly put it into action at work. New ways of thinking about attacking problems, hearing about things that will most likely become issues later, in my opinion is invaluable much for the same reason that subscribing to security mailing lists has value despite the noise, already knowing about that exploit you see on CNN or some of the other online computer site a few days after the code was dropped has value. Frankly being around some of the researchers that have that much "fu" is also valuable because it can show you that what's out in public knowledge about a system is probably not even remotely all that is known or doable with the system not to mention just the inspiration of being around some of these people with that much security brainpower. You wanna get motivated, go listen to Dan Kaminsky talk about bending DNS packets to his will or HD Moore 0wning some un-ownable app, or if packet fu is your thing go listen to Richard Bejtlich or if you are into reversing go listen to Havlar Flake. if that doesnt inspire you to do some work in the home lab or crack a book to be a better security guy/gal, well i dont know what to tell you except to maybe look at why you are in the field.

More random thoughts on the above quote:
At least it can maybe now justify the cost of training you can take at the conference since you usually get access to the talks for free if you took the training. On the other hand, how often has it been that the "obscure non-practical theory/idea" talk actually turned into a huge attack vector? I'm sure the people that first listened to a talk on the supposed vulnerabilities in WEP didn’t really come home with the "practical knowledge" to do anything about it on their networks, but we see later how widespread and dangerous of an attack vector it was. Unfortunately people don’t give a crap about a new vector (it isn’t practical yet) unless the guy is dropping a kiddie friendly tool anyway, then maybe they'll go home and fix or upgrade the network to defend against the attack.

If we do go the "practical knowledge" con route:
Another thing to think about is how do I justify to my boss sending me to a conference where they are going to talk about "practical knowledge" that I can 1) probably get in town from a local training center or 2) from a book for significantly less cost?

Don’t get me wrong, I’m all for a conference where I get something practical out of every talk but I would think its hard to organize a con like that because what might be new information for me might be old news to you. Of course that's probably why there are different tracks and more than one talk going on a time. Valid points though, something for those con organizers to think about at speaker selection time.

Wrap up:
so all that yaking, what's the point? the point, if you just scrolled down to the bottom, is that being able go back and make a change or implement something new on your network after a security con attendance is a poor metric to judge a conference selection of speakers or the value of the conference or of conference attendance. The value of a security conference is more than the talks and beer drinking (both important parts though) that can be done at the conference. The inspiration to do/learn more, exposure to new concepts/methods, and networking with like-minded individuals can pay dividends later as well.


Anonymous said...

I’ve not been to a con, but I have read a good number of their web pages and watched videos from them.

I can see what Don Parker is getting at, but you don’t attend this kind of thing to learn X skill. If you want to get more skills in a certain area, then you go onto a course targeted at that. So if I wanted to get better at firewalling, I would go to my local training place and do a course on that (or get a book) rather than attend a con.

Speaking from my own experience, the con material has been of great use. Yes I cant directly go to my network and apply it. But as CG said, it provides you with ideas. There are too many times to remember that I have read a paper or watched a con video, which was made me think of new ideas or given me a different perspective on my problems and research. The speakers are great for motivating you with your own projects. I’m sure we all have times when we have had enough of our research, when it isn’t work, when you cant break your software’s protection after the 16 millionth time. Watching how the experts do it, seeing that even for them it’s a struggle, but if you stick at it you can get there is a great help. It puts new life into your projects. When then does have an impact on your practical skills and it gets applied to your real world network.

I imagine that it must also be very rewarding to be able to chat with like minded people. I’m guessing that I am not on my own in that here at work there aren’t many people to talk to about InfoSec. Most of the people round me wouldn’t know a netcat from a buffer overflow.

So all in all I think the cons have a place, even if you don’t directly learn some new skill to transfer to your networks. They’re somewhere to learn how the pros do things, get some new ideas, and be generally inspired to get back to the day job and try things.

Richard Bejtlich said...

Good ideas -- thanks for the shout out.

Anonymous said...


First thanks for the shout on the talk. It's very hard to build these talks and put yourself on stage in front of everyone to be judged, and every bit of encouragement counts.

This posting is a very interesting read, because I found myself having this exact conversation arguing the need to balance the "frisbee talks" (as a friend calls them) with more practical talks with some Shmoo folks. The problem is you can't swing the pendulum too far (as you duly noted in your summary).

Cyg, Mikey and I were going for a nice mix, kind of "hey kids here's a cool new stunt that you *can* try at home". It sounds like that came through in it, and I'm working on a new similar toned talk now.

I don't pretend to know what everyone wants, but I've done a lot of talking around DC on this exact subject and am getting a pretty good feel for what apparently a *lot* of people want;
If you're in the DC area, keep your eyes open for a new learning/networking venue.

Anonymous said...

Well the thrust of my column was that these conferences are becoming far too niche like. For the sys admins and security folks out there does a talk on symbian malware really seem relevant? I would argue no. In reality most of these talks have little to do with the day to day machinations of a medium to large corporate network. The talks themselves are indeed high end and cutting edge, just not really relevant to the average IT worker and security monkey who makes up the bulk of the industry today. Were these talks not largely attended primarily by the .gov .mil crowd there would likely be a crap load less attendee's. With that would likely come changes to the con. Not saying I %100 right, but this is also based on feedback from other people who have attended these cons. Anyhow, my two cents worth.

CG said...

"Well the thrust of my column was that these conferences are becoming far too niche like. For the sys admins and security folks out there does a talk on symbian malware really seem relevant? I would argue no."

I would agree whole-heartedly. A sys admin doesnt need to go to a Moible OS conference, but singling out a con as bad based on a few niche talks doesnt make much sense.

Got an example con you are talking about?