How to Analyse Protocols for Potential Vulnerabilities

I was recently asked what is the methodology that I follow to analyse protocols (like IPv6) for potential vulnerabilities, i.e. how can you perform a protocol security assessment. I thought that the answer might be interesting for other guys too and so, I decided to share my experience with you writing a short blog post.


For protocol analysis from a security perspective neither I use any standard fuzzers, as many people would expect, nor I review the source code of a specific implementation, if available (because this last approach would restrict my research to specific implementations only). However, I do agree that in some cases the use of fuzzers could also be useful. What I usually do can be summarised as follows:


  1. First, I study thoroughly the corresponding standards (provided that these are available). Typically, open/public standards are defined in RFCs. Two notes here: First, you usually have to examine more than one RFC – there can be several related ones. All these must be identified, even drafts (because some implementations are described in draft RFCs). Secondly, you will probably need to revisit them several times [digesting them is not an easy process, at least not for me :-) ].

  2. While reading them, try to identify “interesting” cases, that is, potential security issues or testing scenarios. Experience is really important at this step. Use your imagination. Create “what-if” scenarios (e.g. what-if a host constructs a packet not according to the standardised way – of course, you have to make it more specific). You should try to identify either protocol standardisation issues, or cases where a potential implementation may miss an important requirement.

  3. While reading the specs, start building your own tool. You may expect that you will find an open-source available one, but in my experience even if there are available, they can hardly satisfy all your requirements or to address your testing scenarios and needs. You can always use an open source tool and modify it accordingly (contributing also to the corresponding project), but personally I find it easier and more convenient to start from scratch. Doing so you do not inherit potential code weaknesses, but, on the other hand, it requires good programming knowledge from you. Although I love C/C++, for this kind of tasks I prefer Python (and Scapy), just because it is easy :-)

  4. After implementing the basic functionality in your tool, start implementing the features required to address the identified testing scenarios. Try to code your tool in such a way that becomes flexible, i.e. to be able to implement different testing scenarios without requiring too many code changes.

  5. Except from your tool, you will also need a bunch of other tools, at least a network sniffer. I personally prefer wireshark (when I can have a GUI in a system) and tcpdump (for CLI sniffing). Good knowledge of their capabilities is mandatory. Using a sniffer you will be able to identify whether your tool constructs the packets as you expect (or not), how your targets respond, you can store the communication (e.g. in pcap files) for later review, etc.

  6. Finally, you need a lab. Try to incorporate in your testing as many systems as possible, preferably popular ones (either open source, and if possible, commercial too).

  7. Enjoy (testing)! But do not forget to keep detailed notes during the whole process.


I should note that all this aforementioned process is an iterative one, that is, it requires to re-visit the standards, identify new testing cases, implement them, test them, etc. Based on the initial outcome you may have new testing ideas, etc.


Hey, I never said that it is going to be easy. However, if you follow the above, you will end up with:


a. A good knowledge of the tested protocol and its implementation by various systems.


b. A tool (that you can make it available to the community).


c. A good report of a security assessment of a protocol (a write-up is always important). On its own can be a good publication.


d. If you are lucky (or good enough), you may identify a few vulnerabilities, either on specific implementations or on the protocol design itself. In such a case, please disclosure responsibly.


Hope you have fun with this ;-)


Oh, one last requirement. You also need to find ...time. If you know a way how to buy some, please let me know (I have a list of protocols waiting…).


Write a comment

Comments: 0