Over the last few years the number of security tools for Windows and UNIX has risen dramatically, even more surprising is the fact that most of them are freely available on the Internet. I will only cover the free tools since most of the commercial tools are ridiculously expensive, are not open source, and in many cases have been shown to contain major security flaws (like storing passwords in clear text after installation). Any serious cracker/hacker will have these tools at their disposal, so why shouldn't you?
There are several main categories of tools, ones that scan hosts from within that host, ones that scan other hosts and report back variously what OS they are running (using a technique called TCP-IP fingerprinting), services that are available and so on, at the top of the food chain are the intrusion tools that actually attempt to execute exploits, and report back if they worked or not, lastly I include the exploits category, while not strictly an intrusion tool per se they do exist and you should be aware of them.
There are also many free tools and techniques you can use to conduct a self audit and ensure that the systems react as you think they should (we all make errors, but catching them quickly and correcting them is part of what makes a great administrator). Tools such as nmap, nessus, crack, and so forth can be quickly employed to scan your network(s) and host(s), finding any obvious problems quickly. I also suggest you go over your config files every once in a while (for me I try to 'visit' each server once a month, sometimes I discover a small mistake, or something I forgot to set previously). Keeping systems in a relative state of synchronization (I just recently finished moving ALL my customers to Kernel 2.2.x, ipchains) will save you a great deal of time and energy.
Host scanners are software you run locally on the system to probe for problems.
Cops is extremely obsolete and its original home on CERTs ftp site is gone. This is mentioned for historical accuracy only.
Tiger is still under development, albeit slowly, Texas Agricultural and Mechanical University used to require that a UNIX host pass tiger before it was allowed to connect to the network from offsite. You can get it from: ftp://net.tamu.edu/pub/security/TAMU/.
check.pl is a nice Perl program that checks file and directory permissions, and will tell you about any suspicious or bad ones (setuid, setgid, writeable directories, etc). Very useful but it tends to find a lot of false positives. Its available at: http://opop.nols.com/proggie.html.
Network scanners are run from a host and pound away on other machines, looking for open services. If you can find them, chances are an attacker can to. These are generally very useful for ensuring your firewall works.
Strobe is one of the older port scanning tools, quite simply it attempts to connect to various ports on a machine(s) and reports back the result (if any). It is simple to use and very fast, but doesn't have any of the features newer port scanners have. Strobe is available for almost all distributions as part of it, or as a contrib package, the source is available at: ftp://suburbia.net/pub/.
Nmap is a newer and much more fully-featured host scanning tool. It features advanced techniques such as TCP-IP fingerprinting, a method by which the returned TCP-IP packets are examined and the host OS is deduced based on various quirks present in all TCP-IP stacks. Nmap also supports a number of scanning methods from normal TCP scans (simply trying to open a connection as normal) to stealth scanning and half-open SYN scans (great for crashing unstable TCP-IP stacks). This is arguably one of the best port scanning programs available, commercial or otherwise. Nmap is available at: http://www.insecure.org/nmap/index.html. There is also an interesting article available at: http://raven.genome.washington.edu/security/nmap.txt on nmap and using some of its more advanced features.
Portscanner is a nice little portscanner (surprise!) that has varying levels of outputs making it easy to use in scripts and by humans. Its opensource and free to use, you can get it at: http://www.ameth.org/~veilleux/portscan.html.
Queso isnt a scanner per se but it will tell you with a pretty good degree of accuracy what OS a remote host is running. Using a variety of valid and invalid tcp packets to probe the remote host it checks the response against a list of known responses for various operating systems, and will tell you which OS the remote end is running. You can get Queso from: http://www.apostols.org/projectz/queso/.
Intrusion scanners are one evolutionary step up from network scanners. These software packages will actually identify vulnerabilities, and in some cases allow you to actively try and exploit them. If your machines are susceptible to these attacks, you need to start fixing things, as any attacker can get these programs and use them.
Nessus is relatively new but is fast shaping up to be one of the best intrusion scanning tools. It has a client/server architecture, the server currently runs on Linux, FreeBSD, NetBSD and Solaris, clients are available for Linux, Windows and there is a Java client. Communication between the server and client is ciphered for added security all in all a very slick piece of code. Nessus supports port scanning, and attacking, based on IP addresses or host name(s). It can also search through network DNS information and attack related hosts at your bequest. Nessus is relatively slow in attack mode, which is hardly surprising. However it currently has over 200 attacks and a plug-in language so you can write your own. Nessus is available from http://www.nessus.org/.
Saint is the sequel to Satan, a network security scanner made (in)famous by the media a few years ago (there were great worries that bad people would take over the Internet using it). Saint also uses a client/server architecture, but uses a www interface instead of a client program. Saint produces very easy to read and understand output, with security problems graded by priority (although not always correctly) and also supports add-in scanning modules making it very flexible. Saint is available from: http://www.wwdsi.com/saint/.
While not a scanner per se, it is useful for detecting a hosts OS and dealing with a large number of hosts quickly. Cheops is a "network neighborhood" on steroids, it builds a picture of a domain, or IP block, what hosts are running and so on. It is extremely useful for preparing an initial scan as you can locate interesting items (HP printers, Ascend routers, etc) quickly. Cheops is available at: http://www.marko.net/cheops/.
Two simple utilities that scan for ftp servers and mail servers that allow relaying, good for keeping tabs on naughty users installing services they shouldnt (or simply misconfiguring them), available from: http://david.weekly.org/code/.
Security Auditors Research Assistant (SARA) is a tool similar in function to SATAN and Saint. SARA supports multiple threads for faster scans, stores its data in a database for ease of access and generates nice HTML reports. SARA is free for use and is available from: http://home.arc.com/sara/.
BASS is the Bulk Auditing Security Scanner allows you to scan the internet for a variety of well known exploits. It was basically a proof of concept that the Internet is not secure. You can get it from: http://www.securityfocus.com/data/tools/network/bass-1.0.7.tar.gz
There are also a number of programs now that scan firewalls and execute other penetration tests in order to find out how a firewall is configured.
Firewalk is a program that uses a traceroute style of packets to scan a firewall and attempt to deduce the rules in place on that firewall. By sending out packets with various time to lives and seeing where they die or are refused a firewall can be tricked into revealing rules. There is no real defense against this apart from silently denying packets instead of sending a rejection message which hopefully will reveal less. I would advise utilizing this tool against your systems as the results can help you tighten up security. Firewalk is available from: http://www.packetfactory.net/firewalk/.
I won't cover exploits specifically, since there are hundreds if not thousands of them floating around for Linux. I will simply cover the main archival sites.
One of the primary archive sites for exploits, it has almost anything and everything, convenient search engine and generally complete exploits.
Written by Kurt Seifried