Detecting a Breach
Q. What is my role as a systems administrator?
A. To begin, let's be clear that the role of a systems administrator
is very important in the prevention, and detection, of security breaches.
As well, users/customers complement the systems administrator in helping
to detect breaches (for example, unavailable services or a defaced website).
Q. What can I do as a systems administrator right now?
A. The first thing you can do is check your systems. It is important
to not assume that, even though a system currently appears to be running
correctly, it is safe or uncompromised. It is entirely possible that the
system was compromised before you took charge of it, or that it was compromised
in such a way as to not alert you.
Tools such as chckrootkit can test various aspects of the system for
compromise. Windows users can try the Microsoft Baseline Security Analyzer
which will scan for insecure configurations among other things. Another
Microsoft tool is windows update. While this won't test for vulnerabilities
already in the system, it will at least be able to provide one with the
most up to date operating system patches.
Q. What is the most important thing a systems administrator can do?
A. By far, the most important thing a sysadmin can do is patch, Patch,
PATCH. This goes back to being proactive. When one has machines under
his or her charge, they have to make sure that they stay on top of vendor
notices of security problems. Security Focus's Threat Management System
website (http://tms.securityfocus.com)
reports an average latency of 99 days between notification of a vulnerability
and the release of exploit code. This gives you some time, but not much
as many of the high profile exploits of the last year had expliot code
released the same day.
Another very important thing for sysadmins to do is to turn of un-needed
services at the time of machine install. There's simply no reason for
a desktop system to be running a service such as IIS or, with the prevalence
of secure protocols such as ssh, a print server to be running an ftp server.
Q. How should I respond to, and report, a breach?
A. If you suspect that a machine in your charge has been breached, the
first thing you should do is report the incident
to the UCSC Information Systems Security Team.
STOP - DO NOT INVESTIGATE FURTHER ON YOUR OWN
The Security Team will take the responsibility of determining if there
really was a breach (note: it is possible that in your hunt for evidence
of a breach you may destroy evidence which could be needed later in the
event of a formal investigation or possible legal action. It is even possible
that by hunting for evidence, yourself, you may be out of compliance with
UC policy.)
We need to stress the importance of timely notification. We're sure everyone
is aware of the passage of California Senate Bill 1386. As a recap, this
bill states that in the event of a compromise or suspected compromise
of a machine which contains unencrypted personally identifiable information
(PII), the persons whose information resides on that system must be notified
immediately. Immediately isn't defined, but in the case which brought
about this bill, six weeks passed between discovery and notification.
There are some exceptions to the notification, but whether or not your
particular situation falls under one of them is for law enforcement to
decide, not you. Failure to notify the security team of a breach could
potentially leave the university in legal trouble. So let us know.
Q. What other steps can I take to make this easier?
A. If you've done nothing from the start to help yourself detect and
prevent breaches, there are few things that you can do to determine whether
or not your systems have been compromised. These can be quite time consuming
- but there are things you can do that will make a system breach much
more noticeable and hence, discovering and reporting them easier.
- Logging: For Unix systems, enabbling
appropriate logging (for example, sending all critical messages
to one log file, all warnning messages to another, and so on) and then
monitoring those log files is a good start at reducing the signal to
noise ratio that all sysadmins must deal with. From there, a log monitoring
tool such as Logcheck
can be used to parse system log files and send email alerts, further
reducing the ammount of time you must spend in order glean important
information from your busy logs.
- Checksums: Using tools such as Tripwire
or AIDE, which
take cryptographic checksums of system binaries and configuration files
and notify you of differences on subsequent runs can greatly minimize
the ammount of time required to detect intrusions.
Q. What is the UCSC Information Systems Security Team doing to help recognize
breaches?
A. Currently, the UCSC Information Systems Security Team has several
measures in place to help prevent and inform us (and you) about enterprise
level intrusions and intrusion attempts. For example:
- N-IDS: We have network intrusion detection systems (N-IDS)
that look at network traffic for malicious traffic and send notifications
if they encounter such.
- H-IDS: We are running host based intrusion detection systems
on various systems.
- Routers: We enforce various access controls and filtering on
our routers.
- Netflows: We analyze network flows for both detection and investigation.
- Firewalls: We provide firewalling services for systems co-located
within the Data Center (from architecture and design to implementation,
monitoring and management).
In the event that we become aware of an incident, either external or
internal, we coordinate with the sysadmins in charge of the campus machines
in question in order to determine the extent of the breach if any as well
as come up with a plan to mitigate the damage and fix the hole.
|