13 February 2014

Network Interface Card SSH Rootkit

Related papers by Arrigo Triulzi: http://www.alchemistowl.org/arrigo/

Project Maux Mk.II http://t.co/h1gDtV4Vlr

The Jedi Packet Trick takes over the Deathstar http://t.co/ENlITkJEoX

Project Booshoo or the Emporer's Modified Mind https://t.co/33trlpJkFG

Date: Thu, 13 Feb 2014 09:19:13 +0100
From: Matej Kovacic <matej.kovacic[at]owca.info>
To: cypherpunks[at]cpunks.org
Subject: Re: Snowden and Compilers

> It was demonstrated well before then, Arrigo Triulzi had demonstrated running
> an SSH server inside a NIC several years earlier.

Below is a text of interview with Triulzi, from 2009. Website where this was published has some technical problems (only Slovenian version is published, but English will be recovered in a day or two).

Meanwhile, web.archive version is available:

https://web.archive.org/web/20090929091150/http://slo-tech.com/clanki/09010/en

Slovenian version (https://slo-tech.com/clanki/09010/) has a screenschoot of NIC SSH in action.

_____

"All your firmware are belong to us"

Matej Kovacic :: 26 september 2009

Interview with independent security and networking consultant Arrigo Triulzi

Arrigo Triulzi is an independent security and networking consultant, working out of Geneva in Switzerland. He is a mathematician by training but has been a consultant for over 20 years. His hobby is firmware research. In November 2008 he had a presentation about project Maux -- his research about firmware rootkits.

Since firmware rootkits and exploits are interesting area of research, we decided to make a short interview with him to present this topic to our readers.

Matej Kovacic: First question -- could you describe rootkits for our readers in a few words?

Arrigo Triulzi: A rootkit is normally defined as a collection of tools which allow the remote control of a system. They generally consist of at least a sniffer, a password cracker and a backdoor.

Matej Kovacic: What is the difference between software and hardware rootkits?

Arrigo Triulzi: A software rootkit is tailored to an operating system, often a specific version of an operating system, a hardware rootkit is tailored to the hardware of a specific system and is therefore operating system independent. A reinstall of software on a system infected with a hardware rootkit has no effect on the rootkit's operation.

Matej Kovacic: Last year you had the presentation about malicious code, stored on a network card. Could you describe what is the general idea of hardware rootkits and what exactly have you done?

Arrigo Triulzi: The idea behind hardware rootkits (see John Heasman's ACPI work, Jason Larson's work on in-memory rootkits, earlier work by the Austrian TESO group and Rutkowska's recent work on AMT) is simply to have rootkits which are operating system independent and, in some cases, hardware independent.

Hardware independence is given by the fact that a rootkit running in a network card works independently of where that network card is used: if it is in an IBM Power workstation, a Dell PC, a high-end switch or a MIPS-based SOHO router. As long as the NIC chip is the same then it will work.

What I have done is to modify the NIC firmware in such a way that it can react to external commands and, in association with other equipment on the motherboard (the video card, specifically), provide a remote shell.

This remote shell is capable of viewing the system memory from which you can extract whatever is of interest (passwords stored in clear, keyboard buffers, graphics buffers, etc.).

An extension of this work is to provide a direct communications channel between two NICs thereby providing a bypass in a firewall which is totally invisible to the firewall's operating system.

Matej Kovacic: How were you able to load your firmware into the card? You need some special hardware device or you can do it simply from the computer? Is there any protection, like digital signatures of new firmware?

Arrigo Triulzi: Well, as all research projects it evolved from using the flash update program which came with the NIC and simply feeding it my modified firmware to using more sophisticated techniques like discovering a remote update capability. The current version simply sends the appropriate packets to vulnerable cards and updates them.

Different manufacturers use different layers of protection and it is a general observation that the cheaper the NIC the worse the protection is. Quite a few high-end cards make use of digital signatures to verify that the new firmware is indeed from the manufacturer.

Matej Kovacic: How hard was to develop such a code, what kind of a knowledge and equipment do you need?

Arrigo Triulzi: It all started out of curiosity and took about two years of working when I had time to spare. The knowledge I built as I went along and, for the hardware, by asking my father who used to design computers.

Matej Kovacic: Approximate, about how many man-hours? By your opinion, how much would need some well funded criminal organization?

Arrigo Triulzi: Well, I would say that it took me approximately 500 man-hours to get to the stage of having a working "shell" (it cannot really be called a real shell as the commands are very limited). I doubt that a well-funded organization would need much more time than I needed but it should be noted that the ROI does not make the endeavour worthwhile: while the current mechanisms work (phishing, viruses, etc.) there is no point in spending time and money to go down the hardware route.

Matej Kovacic: There are several producers of network cards. How many chipsets are in use? Rootkits for every branch should probably differ. How much? Would it be possible to develop generic ethernet card rootkit?

Arrigo Triulzi: As I describe in my paper the marketing department is your worst enemy: there are literally thousands of variants, many of which are only ever documented in the OEM datasheets, and often little changes (deltas) in the NIC production line cause my modified firmware to fail.

A perfect example: I bought two 10-pack of allegedly identical NICs (identical model numbers) and the production batches differed enough that I needed to have two separate versions. The changes were minor but it was only when I looked carefully at the main chip on the NIC that I realized that they were different revisions of the same chip.

Now, can a generic NIC rootkit be developed? I don't know, it might be possible to write a NIC rootkit which works across families in the same way that a kernel can run (at times suboptimally) on CPUs from Pentium through Core i7 but I have not really looked at it.

Matej Kovacic: If you can run a custom code on a network card firmware, you have direct memory access to RAM...

Arrigo Triulzi: Yes, you do, via DMA.

Matej Kovacic: Which means you can easily read and write memory without CPU knowing anything about it... What are the other security impacts of this and of your research?

Arrigo Triulzi: Well, the first problem is that it is invisible except for the traffic on the network side - if you want to take data out then you must pass it over the network and this means that it could be detected. On the other hand the operating system has no hope unless it has some way of comparing the current firmware with the original one.

The security impacts are a simple extrapolation of the above: you can read (and write) data into any system in a way which is currently totally undetectable.

Matej Kovacic: How to detect such attack? It seems software techniques are long time not enough to keep us secure...

Arrigo Triulzi: The only way would be trusted computing if implemented properly and without the DRM halo which it normally carries: it would mean that the firmware on the NIC is trusted by the system and therefore any modification would be detected and prevented.

There is probably no easy way to detect it in software - a simply firmware comparison could be tricked by keeping a copy of the parts which were modified on the firmware and returning those to anyone trying to download the firmware for verification.

Matej Kovacic: How exactly would be possible to detect if firmare is correct? TPM chip on every device?

Arrigo Triulzi: Well, if we were to implement TPM in such a way that every component of a system is verified for both security and integrity (it would be nice to know if your NIC is about to fail, independently of the security of the device). This makes the boot process complicated because you start having issues with chicken & eggs: what part of the hardware needs to be powered up and in what order? How do you execute correctly and, more importantly, securely, a WOL (wake-on-LAN) packet?

If you start thinking about it you realize that it is not a trivial issue at all: if you need to process a WOL packet it means that you have to have the NIC wake up first, but then how does the TPM verify that the WOL packet has not tampered the NIC? Do you do it later in the process? But then how do you know that the NIC has not tampered with other parts of the system or, worse, is replying pretending to be other devices on the PCI bus to the TPM queries? The paranoid can head for the hills now.

Matej Kovacic: What if someone installs malicious code in the factory?

Arrigo Triulzi: This is obviously a nightmare -- you get a pre-installed botnet with your Christmas PC purchase. It is one of the examples I always use: someone installing modified NIC firmware at Dell just before the Christmas rush, Dell ships loads of PCs and mid-January we have a gigantic botnet distributed around the planet by DHL. No chance of seeing the infection vector because it never touches the Internet until the botnet is unleashed. See it as an out-of-band infection mechanism.

Matej Kovacic: How much memory does have network card? How big can be malicious firmware?

Arrigo Triulzi: Very very little. This is why I had to branch out to the GPU (video card) to find memory to put my shell in. I only really use the smallest amount of RAM and firmware space on the NIC device, everything else is done via DMA on the video card. In my first paper on NIC firmware modification (presented at a closed conference) I gave the figure of approximately 5s of sniffer data being held on the device before the RAM filled up.

Matej Kovacic: Have you heard of Phenoelit research of security of IOS operating system? Last year they demonstrated an interesting attack on Cisco routers – they sent one special packet through router and were able - for instance - to change firewall rules.

It seems it would be possible to bypass the network filters by smuggling network card into the company, penetrate network routers from inside, get internet access to compromised ethernet card and... the sky is the limit?

Arrigo Triulzi: Indeed I have read Phenoelit's research and rather than smuggling a NIC into the company a better trick would be to take over the firewall directly: if the firewall has NICs which are vulnerable you can use something I call the "Jedi Packet Trick" to take over the external NIC, then via the PCI bus infect the internal NIC and pass packets directly between the two NICs. The name obviously comes from the Jedi Mind Trick used to bypass the Imperial Stormtroopers ...

Matej Kovacic: Last month we saw a malicious code could be also runned on a Mac keyboard. Joanna Rutkowska and her team had shown how vulnerable are hypervisors. Which hardware could also be exploited?

Arrigo Triulzi: Well, anything with a CPU: from the service processors (IPMI, AMT, HP iLO) to SATA disks, to SCSI controllers, to video cards (these I've already done some work on), network cards, etc.

Matej Kovacic: Yes, in your presentation you mentioned, you are able to connect to video cards through network card (via PCI-to-PCI transport) and run a malicious code there. This is much more powerful, because video cards have much more memory and GPU has much more computing power... Could you explain this in more detail?

Arrigo Triulzi: Quite simply the NIC is not powerful enough to do much more than take packets off the wire and route them to a different location over the PCI bus. You therefore need somewhere to run more complex code and the obvious answer is (avoiding software, of course) the GPU: the current crop from both ATI and nVidia are extremely powerful and nVidia comes with a good open-source development kit.

So the packets come into the NIC, they are checked to see if they are to be forwarded to the video card or are legitimate packets, they get to the video card which processes the command and sends the reply back through the same route.

Matej Kovacic: We haven't seen much security research about hardware based attacks (comparative to software based attacks). Is there any security testing of a new hadrware?

Arrigo Triulzi: Not really, no. At the moment hardware security rests with manufacturers. I suspect that governments will test hardware security for their more sensitive applications.

Matej Kovacic: Maybe some advice what to do if you are really paranoid about security?

Arrigo Triulzi: As things stand I would recommend using pen and paper... More seriously now: hardware rootkits are very sexy but hardly what is needed these days. What is the point of a sophisticated hardware rootkit when you can gain control of a user's machine by simply sending a malicious image as part of a website or a malicious attachment with an e-mail? At the moment the return on investment for hardware rootkits do not make them viable so they are simply not going to be used except where they are really needed which, in my opinion, is at the level of serious industrial espionage. So the answer is that you had better concern yourself with securing what you already use and make sure that your online behaviour is not reckless, mail programs should come with three to five layers of big red dialog boxes before they let you open an attachment of any kind...

Matej Kovacic: I agree. But if you are a big company or government organization, this things can become important. And then it is only a question of trust. I mean, at the end of the day, you have to trust someone – your software, your OS, your hardware. Who do you trust?

Arrigo Triulzi: As I mentioned earlier the more expensive cards do come with signed firmware. If you decide to purchase the very cheapest equipment then chances are that one of the areas the company didn’t invest in is security. If you are a government organization you often already have a source license for Windows, for example, and it is just routine for approved vendors to provide hardware specifications including the firmware update mechanism. You simply need to check that it uses signed firmware and then verify the claim.
In many ways it is the usual story: "you get what you pay for".

Matej Kovacic: Thank you very much.