13 February 2014
Network Interface Card SSH Rootkit
Related papers by Arrigo Triulzi:
Project Maux Mk.II
The Jedi Packet Trick takes over the Deathstar
Project Booshoo or the Emporer's Modified
Date: Thu, 13 Feb 2014 09:19:13 +0100
From: Matej Kovacic <matej.kovacic[at]owca.info>
Subject: Re: Snowden and Compilers
> It was demonstrated well before then, Arrigo Triulzi had demonstrated
> an SSH server inside a NIC several years earlier.
Below is a text of interview with Triulzi, from 2009. Website where this
was published has some technical problems (only Slovenian version is published,
but English will be recovered in a day or two).
Meanwhile, web.archive version is available:
has a screenschoot of NIC SSH in action.
your firmware are belong to us"
Matej Kovacic :: 26 september 2009
Interview with independent security and networking consultant Arrigo
Triulzi is an independent security and networking consultant, working
out of Geneva in Switzerland. He is a mathematician by training but has been
a consultant for over 20 years. His hobby is firmware research. In November
2008 he had a presentation about
Maux -- his research about firmware rootkits.
Since firmware rootkits and exploits are interesting area of research, we
decided to make a short interview with him to present this topic to our readers.
Matej Kovacic: First question -- could you describe
rootkits for our readers in a few words?
Arrigo Triulzi: A rootkit is normally defined as a collection
of tools which allow the remote control of a system. They generally consist
of at least a sniffer, a password cracker and a backdoor.
Matej Kovacic: What is the difference between software
and hardware rootkits?
Arrigo Triulzi: A software rootkit is tailored to an operating
system, often a specific version of an operating system, a hardware rootkit
is tailored to the hardware of a specific system and is therefore operating
system independent. A reinstall of software on a system infected with a hardware
rootkit has no effect on the rootkit's operation.
Matej Kovacic: Last year you had the presentation about
malicious code, stored on a network card. Could you describe what is the
general idea of hardware rootkits and what exactly have you done?
Arrigo Triulzi: The idea behind hardware rootkits (see John
Heasman's ACPI work, Jason Larson's work on in-memory rootkits, earlier work
by the Austrian TESO group and Rutkowska's recent work on AMT) is simply
to have rootkits which are operating system independent and, in some cases,
Hardware independence is given by the fact that a rootkit running in a network
card works independently of where that network card is used: if it is in
an IBM Power workstation, a Dell PC, a high-end switch or a MIPS-based SOHO
router. As long as the NIC chip is the same then it will work.
What I have done is to modify the NIC firmware in such a way that it can
react to external commands and, in association with other equipment on the
motherboard (the video card, specifically), provide a remote shell.
This remote shell is capable of viewing the system memory from which you
can extract whatever is of interest (passwords stored in clear, keyboard
buffers, graphics buffers, etc.).
An extension of this work is to provide a direct communications channel between
two NICs thereby providing a bypass in a firewall which is totally invisible
to the firewall's operating system.
Matej Kovacic: How were you able to load your firmware
into the card? You need some special hardware device or you can do it simply
from the computer? Is there any protection, like digital signatures of new
Arrigo Triulzi: Well, as all research projects it evolved
from using the flash update program which came with the NIC and simply feeding
it my modified firmware to using more sophisticated techniques like discovering
a remote update capability. The current version simply sends the appropriate
packets to vulnerable cards and updates them.
Different manufacturers use different layers of protection and it is a general
observation that the cheaper the NIC the worse the protection is. Quite a
few high-end cards make use of digital signatures to verify that the new
firmware is indeed from the manufacturer.
Matej Kovacic: How hard was to develop such a code,
what kind of a knowledge and equipment do you need?
Arrigo Triulzi: It all started out of curiosity and took
about two years of working when I had time to spare. The knowledge I built
as I went along and, for the hardware, by asking my father who used to design
Matej Kovacic: Approximate, about how many man-hours?
By your opinion, how much would need some well funded criminal
Arrigo Triulzi: Well, I would say that it took me approximately
500 man-hours to get to the stage of having a working "shell" (it cannot
really be called a real shell as the commands are very limited). I doubt
that a well-funded organization would need much more time than I needed but
it should be noted that the ROI does not make the endeavour worthwhile: while
the current mechanisms work (phishing, viruses, etc.) there is no point in
spending time and money to go down the hardware route.
Matej Kovacic: There are several producers of network
cards. How many chipsets are in use? Rootkits for every branch should probably
differ. How much? Would it be possible to develop generic ethernet card
Arrigo Triulzi: As I describe in my paper the marketing
department is your worst enemy: there are literally thousands of variants,
many of which are only ever documented in the OEM datasheets, and often little
changes (deltas) in the NIC production line cause my modified firmware to
A perfect example: I bought two 10-pack of allegedly identical NICs (identical
model numbers) and the production batches differed enough that I needed to
have two separate versions. The changes were minor but it was only when I
looked carefully at the main chip on the NIC that I realized that they were
different revisions of the same chip.
Now, can a generic NIC rootkit be developed? I don't know, it might be possible
to write a NIC rootkit which works across families in the same way that a
kernel can run (at times suboptimally) on CPUs from Pentium through Core
i7 but I have not really looked at it.
Matej Kovacic: If you can run a custom code on a network
card firmware, you have direct memory access to RAM...
Arrigo Triulzi: Yes, you do, via DMA.
Matej Kovacic: Which means you can easily read and write
memory without CPU knowing anything about it... What are the other security
impacts of this and of your research?
Arrigo Triulzi: Well, the first problem is that it is invisible
except for the traffic on the network side - if you want to take data out
then you must pass it over the network and this means that it could be detected.
On the other hand the operating system has no hope unless it has some way
of comparing the current firmware with the original one.
The security impacts are a simple extrapolation of the above: you can read
(and write) data into any system in a way which is currently totally
Matej Kovacic: How to detect such attack? It seems software
techniques are long time not enough to keep us secure...
Arrigo Triulzi: The only way would be trusted computing
if implemented properly and without the DRM halo which it normally carries:
it would mean that the firmware on the NIC is trusted by the system and therefore
any modification would be detected and prevented.
There is probably no easy way to detect it in software - a simply firmware
comparison could be tricked by keeping a copy of the parts which were modified
on the firmware and returning those to anyone trying to download the firmware
Matej Kovacic: How exactly would be possible to detect
if firmare is correct? TPM chip on every device?
Arrigo Triulzi: Well, if we were to implement TPM in such
a way that every component of a system is verified for both security and
integrity (it would be nice to know if your NIC is about to fail, independently
of the security of the device). This makes the boot process complicated because
you start having issues with chicken & eggs: what part of the hardware
needs to be powered up and in what order? How do you execute correctly and,
more importantly, securely, a WOL (wake-on-LAN) packet?
If you start thinking about it you realize that it is not a trivial issue
at all: if you need to process a WOL packet it means that you have to have
the NIC wake up first, but then how does the TPM verify that the WOL packet
has not tampered the NIC? Do you do it later in the process? But then how
do you know that the NIC has not tampered with other parts of the system
or, worse, is replying pretending to be other devices on the PCI bus to the
TPM queries? The paranoid can head for the hills now.
Matej Kovacic: What if someone installs malicious code
in the factory?
Arrigo Triulzi: This is obviously a nightmare -- you get
a pre-installed botnet with your Christmas PC purchase. It is one of the
examples I always use: someone installing modified NIC firmware at Dell just
before the Christmas rush, Dell ships loads of PCs and mid-January we have
a gigantic botnet distributed around the planet by DHL. No chance of seeing
the infection vector because it never touches the Internet until the botnet
is unleashed. See it as an out-of-band infection mechanism.
Matej Kovacic: How much memory does have network card?
How big can be malicious firmware?
Arrigo Triulzi: Very very little. This is why I had to branch
out to the GPU (video card) to find memory to put my shell in. I only really
use the smallest amount of RAM and firmware space on the NIC device, everything
else is done via DMA on the video card. In my first paper on NIC firmware
modification (presented at a closed conference) I gave the figure of
approximately 5s of sniffer data being held on the device before the RAM
Matej Kovacic: Have you heard of Phenoelit research of
security of IOS operating system?
year they demonstrated an interesting attack on Cisco routers
one special packet through router and were able - for instance - to change
It seems it would be possible to bypass the network filters by smuggling
network card into the company, penetrate network routers from inside, get
internet access to compromised ethernet card and... the sky is the limit?
Arrigo Triulzi: Indeed I have read Phenoelit's research
and rather than smuggling a NIC into the company a better trick would be
to take over the firewall directly: if the firewall has NICs which are vulnerable
you can use something I call the "Jedi Packet Trick" to take over the external
NIC, then via the PCI bus infect the internal NIC and pass packets directly
between the two NICs. The name obviously comes from the Jedi Mind Trick used
to bypass the Imperial Stormtroopers ...
Matej Kovacic: Last month we saw a malicious code could
be also runned on a Mac keyboard. Joanna Rutkowska and her team had shown
how vulnerable are hypervisors. Which hardware could also be exploited?
Arrigo Triulzi: Well, anything with a CPU: from the service
processors (IPMI, AMT, HP iLO) to SATA disks, to SCSI controllers, to video
cards (these I've already done some work on), network cards, etc.
Matej Kovacic: Yes, in your presentation you mentioned,
you are able to connect to video cards through network card (via PCI-to-PCI
transport) and run a malicious code there. This is much more powerful, because
video cards have much more memory and GPU has much more computing power...
Could you explain this in more detail?
Arrigo Triulzi: Quite simply the NIC is not powerful enough
to do much more than take packets off the wire and route them to a different
location over the PCI bus. You therefore need somewhere to run more complex
code and the obvious answer is (avoiding software, of course) the GPU: the
current crop from both ATI and nVidia are extremely powerful and nVidia comes
with a good open-source development kit.
So the packets come into the NIC, they are checked to see if they are to
be forwarded to the video card or are legitimate packets, they get to the
video card which processes the command and sends the reply back through the
Matej Kovacic: We haven't seen much security research
about hardware based attacks (comparative to software based attacks). Is
there any security testing of a new hadrware?
Arrigo Triulzi: Not really, no. At the moment hardware security
rests with manufacturers. I suspect that governments will test hardware security
for their more sensitive applications.
Matej Kovacic: Maybe some advice what to do if you are
really paranoid about security?
Arrigo Triulzi: As things stand I would recommend using
pen and paper... More seriously now: hardware rootkits are very sexy but
hardly what is needed these days. What is the point of a sophisticated hardware
rootkit when you can gain control of a user's machine by simply sending a
malicious image as part of a website or a malicious attachment with an e-mail?
At the moment the return on investment for hardware rootkits do not make
them viable so they are simply not going to be used except where they are
really needed which, in my opinion, is at the level of serious industrial
espionage. So the answer is that you had better concern yourself with securing
what you already use and make sure that your online behaviour is not reckless,
mail programs should come with three to five layers of big red dialog boxes
before they let you open an attachment of any kind...
Matej Kovacic: I agree. But if you are a big company
or government organization, this things can become important. And then it
is only a question of trust. I mean, at the end of the day, you have to trust
someone â your software, your OS, your hardware. Who do
Arrigo Triulzi: As I mentioned earlier the more expensive
cards do come with signed firmware. If you decide to purchase the very cheapest
equipment then chances are that one of the areas the company
didnât invest in is security. If you are a government
organization you often already have a source license for Windows, for example,
and it is just routine for approved vendors to provide hardware specifications
including the firmware update mechanism. You simply need to check that it
uses signed firmware and then verify the claim.
In many ways it is the usual story: "you get what you pay for".
Matej Kovacic: Thank you very much.