inital public release

.gitignore

@@ -0,0 +1,6 @@
+*.o
+*.a
+*.pyc
+*~
+\#*
+src/zmap

AUTHORS

@@ -0,0 +1,3 @@
+Zakir Durumeric <zakird@umich.edu>
+J. Alex Halderman <jhalderm@umich.edu>
+Eric Wustrow <ewust@umich.edu>

CHANGELOG

@@ -0,0 +1,2 @@
+1.0.0 	2013-8-16
+Initial public release.

INSTALL

@@ -0,0 +1,25 @@
+ZMap is designed to run on GNU/Linux systems and can be built with
+most recent versions of gcc.  ZMap requires GMP, a free library for
+arbitrary precision arithmetic, gengetopt, and libpcap.
+
+These packages can be installed on Debian-based systems by running:
+
+  sudo apt-get install libgmp3-dev gengetopt libpcap-dev
+ 
+or on RHEL- and Fedora-based systems by running:
+
+  sudo yum install gmp gmp-devel gengetopt libpcap-devel
+
+Once these prerequisites have been installed, ZMap can be installed by running:
+
+  cd src
+  make
+
+followed by:
+
+  sudo make install
+
+Redis support is not enabled by default.  If you are want to use ZMap
+with Redis, you will first need to install Hiredis.  Then, rebuild
+ZMap with the command "make REDIS=true".
+

LICENSE

@@ -0,0 +1,177 @@
+
+                                 Apache License
+                           Version 2.0, January 2004
+                        http://www.apache.org/licenses/
+
+   TERMS AND CONDITIONS FOR USE, REPRODUCTION, AND DISTRIBUTION
+
+   1. Definitions.
+
+      "License" shall mean the terms and conditions for use, reproduction,
+      and distribution as defined by Sections 1 through 9 of this document.
+
+      "Licensor" shall mean the copyright owner or entity authorized by
+      the copyright owner that is granting the License.
+
+      "Legal Entity" shall mean the union of the acting entity and all
+      other entities that control, are controlled by, or are under common
+      control with that entity. For the purposes of this definition,
+      "control" means (i) the power, direct or indirect, to cause the
+      direction or management of such entity, whether by contract or
+      otherwise, or (ii) ownership of fifty percent (50%) or more of the
+      outstanding shares, or (iii) beneficial ownership of such entity.
+
+      "You" (or "Your") shall mean an individual or Legal Entity
+      exercising permissions granted by this License.
+
+      "Source" form shall mean the preferred form for making modifications,
+      including but not limited to software source code, documentation
+      source, and configuration files.
+
+      "Object" form shall mean any form resulting from mechanical
+      transformation or translation of a Source form, including but
+      not limited to compiled object code, generated documentation,
+      and conversions to other media types.
+
+      "Work" shall mean the work of authorship, whether in Source or
+      Object form, made available under the License, as indicated by a
+      copyright notice that is included in or attached to the work
+      (an example is provided in the Appendix below).
+
+      "Derivative Works" shall mean any work, whether in Source or Object
+      form, that is based on (or derived from) the Work and for which the
+      editorial revisions, annotations, elaborations, or other modifications
+      represent, as a whole, an original work of authorship. For the purposes
+      of this License, Derivative Works shall not include works that remain
+      separable from, or merely link (or bind by name) to the interfaces of,
+      the Work and Derivative Works thereof.
+
+      "Contribution" shall mean any work of authorship, including
+      the original version of the Work and any modifications or additions
+      to that Work or Derivative Works thereof, that is intentionally
+      submitted to Licensor for inclusion in the Work by the copyright owner
+      or by an individual or Legal Entity authorized to submit on behalf of
+      the copyright owner. For the purposes of this definition, "submitted"
+      means any form of electronic, verbal, or written communication sent
+      to the Licensor or its representatives, including but not limited to
+      communication on electronic mailing lists, source code control systems,
+      and issue tracking systems that are managed by, or on behalf of, the
+      Licensor for the purpose of discussing and improving the Work, but
+      excluding communication that is conspicuously marked or otherwise
+      designated in writing by the copyright owner as "Not a Contribution."
+
+      "Contributor" shall mean Licensor and any individual or Legal Entity
+      on behalf of whom a Contribution has been received by Licensor and
+      subsequently incorporated within the Work.
+
+   2. Grant of Copyright License. Subject to the terms and conditions of
+      this License, each Contributor hereby grants to You a perpetual,
+      worldwide, non-exclusive, no-charge, royalty-free, irrevocable
+      copyright license to reproduce, prepare Derivative Works of,
+      publicly display, publicly perform, sublicense, and distribute the
+      Work and such Derivative Works in Source or Object form.
+
+   3. Grant of Patent License. Subject to the terms and conditions of
+      this License, each Contributor hereby grants to You a perpetual,
+      worldwide, non-exclusive, no-charge, royalty-free, irrevocable
+      (except as stated in this section) patent license to make, have made,
+      use, offer to sell, sell, import, and otherwise transfer the Work,
+      where such license applies only to those patent claims licensable
+      by such Contributor that are necessarily infringed by their
+      Contribution(s) alone or by combination of their Contribution(s)
+      with the Work to which such Contribution(s) was submitted. If You
+      institute patent litigation against any entity (including a
+      cross-claim or counterclaim in a lawsuit) alleging that the Work
+      or a Contribution incorporated within the Work constitutes direct
+      or contributory patent infringement, then any patent licenses
+      granted to You under this License for that Work shall terminate
+      as of the date such litigation is filed.
+
+   4. Redistribution. You may reproduce and distribute copies of the
+      Work or Derivative Works thereof in any medium, with or without
+      modifications, and in Source or Object form, provided that You
+      meet the following conditions:
+
+      (a) You must give any other recipients of the Work or
+          Derivative Works a copy of this License; and
+
+      (b) You must cause any modified files to carry prominent notices
+          stating that You changed the files; and
+
+      (c) You must retain, in the Source form of any Derivative Works
+          that You distribute, all copyright, patent, trademark, and
+          attribution notices from the Source form of the Work,
+          excluding those notices that do not pertain to any part of
+          the Derivative Works; and
+
+      (d) If the Work includes a "NOTICE" text file as part of its
+          distribution, then any Derivative Works that You distribute must
+          include a readable copy of the attribution notices contained
+          within such NOTICE file, excluding those notices that do not
+          pertain to any part of the Derivative Works, in at least one
+          of the following places: within a NOTICE text file distributed
+          as part of the Derivative Works; within the Source form or
+          documentation, if provided along with the Derivative Works; or,
+          within a display generated by the Derivative Works, if and
+          wherever such third-party notices normally appear. The contents
+          of the NOTICE file are for informational purposes only and
+          do not modify the License. You may add Your own attribution
+          notices within Derivative Works that You distribute, alongside
+          or as an addendum to the NOTICE text from the Work, provided
+          that such additional attribution notices cannot be construed
+          as modifying the License.
+
+      You may add Your own copyright statement to Your modifications and
+      may provide additional or different license terms and conditions
+      for use, reproduction, or distribution of Your modifications, or
+      for any such Derivative Works as a whole, provided Your use,
+      reproduction, and distribution of the Work otherwise complies with
+      the conditions stated in this License.
+
+   5. Submission of Contributions. Unless You explicitly state otherwise,
+      any Contribution intentionally submitted for inclusion in the Work
+      by You to the Licensor shall be under the terms and conditions of
+      this License, without any additional terms or conditions.
+      Notwithstanding the above, nothing herein shall supersede or modify
+      the terms of any separate license agreement you may have executed
+      with Licensor regarding such Contributions.
+
+   6. Trademarks. This License does not grant permission to use the trade
+      names, trademarks, service marks, or product names of the Licensor,
+      except as required for reasonable and customary use in describing the
+      origin of the Work and reproducing the content of the NOTICE file.
+
+   7. Disclaimer of Warranty. Unless required by applicable law or
+      agreed to in writing, Licensor provides the Work (and each
+      Contributor provides its Contributions) on an "AS IS" BASIS,
+      WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or
+      implied, including, without limitation, any warranties or conditions
+      of TITLE, NON-INFRINGEMENT, MERCHANTABILITY, or FITNESS FOR A
+      PARTICULAR PURPOSE. You are solely responsible for determining the
+      appropriateness of using or redistributing the Work and assume any
+      risks associated with Your exercise of permissions under this License.
+
+   8. Limitation of Liability. In no event and under no legal theory,
+      whether in tort (including negligence), contract, or otherwise,
+      unless required by applicable law (such as deliberate and grossly
+      negligent acts) or agreed to in writing, shall any Contributor be
+      liable to You for damages, including any direct, indirect, special,
+      incidental, or consequential damages of any character arising as a
+      result of this License or out of the use or inability to use the
+      Work (including but not limited to damages for loss of goodwill,
+      work stoppage, computer failure or malfunction, or any and all
+      other commercial damages or losses), even if such Contributor
+      has been advised of the possibility of such damages.
+
+   9. Accepting Warranty or Additional Liability. While redistributing
+      the Work or Derivative Works thereof, You may choose to offer,
+      and charge a fee for, acceptance of support, warranty, indemnity,
+      or other liability obligations and/or rights consistent with this
+      License. However, in accepting such obligations, You may act only
+      on Your own behalf and on Your sole responsibility, not on behalf
+      of any other Contributor, and only if You agree to indemnify,
+      defend, and hold each Contributor harmless for any liability
+      incurred by, or claims asserted against, such Contributor by reason
+      of your accepting any such warranty or additional liability.
+
+   END OF TERMS AND CONDITIONS

README

@@ -0,0 +1,25 @@
+ZMap is a fast network scanner designed for Internet-wide network surveys. On a
+typical desktop computer with a gigabit Ethernet connection, ZMap is capable
+scanning the entire public IPv4 address space in under 45 minutes.
+
+While previous network tools have been designed to scan small network segments,
+ZMap is specifically architected to scan the entire address space.  It is built
+in a modular manner in order to allow incorporation with other network survey
+tools.  ZMap operates on GNU/Linux and supports TCP SYN and ICMP echo request
+scanning out of the box.
+
+Documentation and examples can be found at https://zmap.io/.
+
+-- 
+
+ZMap Copyright 2013 Regents of the University of Michigan 
+
+Licensed under the Apache License, Version 2.0 (the "License"); you may not use
+this file except in compliance with the License. You may obtain a copy of the
+License at http://www.apache.org/licenses/LICENSE-2.0
+
+Unless required by applicable law or agreed to in writing, software distributed
+under the License is distributed on an "AS IS" BASIS, WITHOUT WARRANTIES OR
+CONDITIONS OF ANY KIND, either express or implied. See LICENSE for the specific
+language governing permissions and limitations under the License.
+

conf/blacklist.conf

@@ -0,0 +1,25 @@
+# From IANA IPv4 Special-Purpose Address Registry
+# http://www.iana.org/assignments/iana-ipv4-special-registry/iana-ipv4-special-registry.xhtml
+# Updated 2013-05-22
+
+0.0.0.0/8           # RFC1122: "This host on this network"
+10.0.0.0/8          # RFC1918: Private-Use
+100.64.0.0/10       # RFC6598: Shared Address Space
+127.0.0.0/8         # RFC1122: Loopback
+169.254.0.0/16      # RFC3927: Link Local
+172.16.0.0/12       # RFC1918: Private-Use
+192.0.0.0/24        # RFC6890: IETF Protocol Assignments
+192.0.2.0/24        # RFC5737: Documentation (TEST-NET-1)
+192.88.99.0/24      # RFC3068: 6to4 Relay Anycast
+192.168.0.0/16      # RFC1918: Private-Use
+192.18.0.0/15       # RFC2544: Benchmarking
+198.51.100.0/24     # RFC5737: Documentation (TEST-NET-2)
+203.0.113.0/24      # RFC5737: Documentation (TEST-NET-3)
+240.0.0.0/4         # RFC1112: Reserved
+255.255.255.255/32  # RFC0919: Limited Broadcast
+
+# From IANA Multicast Address Space Registry
+# http://www.iana.org/assignments/multicast-addresses/multicast-addresses.xhtml
+# Updated 2013-06-25
+
+224.0.0.0/4         # RFC5771: Multicast/Reserved

conf/zmap.conf

@@ -0,0 +1,22 @@
+### Probe Module to use
+#probe-module tcp_synscan
+
+
+### Destination port to scan
+#target-port 443
+
+### Scan rate in packets/sec
+#rate 10000
+
+### Scan rate in bandwidth (bits/sec); overrides `rate`
+#bandwidth 1M	# 1mbps
+
+
+### Blacklist file to use. We encourage you to exclude
+### RFC1918, IANA reserved, and multicast networks,
+### in addition to those who have opted out of your
+### network scans.
+blacklist-file "/etc/zmap/blacklist.conf"
+
+### Optionally print a summary at the end
+#summary

examples/banner-grab/Makefile

@@ -0,0 +1,22 @@
+
+
+CFLAGS+=-I../../lib/ -Wall
+LDFLAGS+=-lpcap -levent -levent_extra -lm
+VPATH=../../lib/
+
+# from dpkg-buildflags --get CFLAGS, but use stack-protector-all and fPIC
+GCCHARDENING=-g -O2 -fstack-protector-all --param=ssp-buffer-size=4 -Wformat -Wformat-security -Werror=format-security -fPIC
+# from gpkg-buildflags --get LDFLAGS, + -z,now
+LDHARDENING=-Wl,-Bsymbolic-functions -Wl,-z,relro,-z,now
+
+CFLAGS+=$(GCCHARDENING)
+LDFLAGS+=$(LDHARDENING)
+
+
+all: banner-grab-tcp
+
+banner-grab-tcp: banner-grab-tcp.o logger.o
+	$(CC) $(CFLAGS) $^ -o $@ $(LDFLAGS)
+
+clean:
+	rm -f banner-grab-tcp *.o

examples/banner-grab/README

@@ -0,0 +1,54 @@
+
+TCP Banner Grab
+======
+
+This utility will connect (TCP) to ip addresses provide over stdin, optionally
+send them a small message, and wait for their response. The response is then
+printed along with their IP address on stdout. Status messages appear on stderr.
+
+
+USING:
+-----
+make
+#echo -e -n "GET / HTTP/1.1\r\nHost: %s\r\n\r\n" > http-req
+zmap -p 80 -N 1000 -o - | ./banner-grab-tcp -p 80 -c 100 -d http-req > http-banners.out
+
+
+OPTIONS:
+-----
+-c, --concurent         Number of connections that can be going on at once.
+                        This, combined with timeouts, will decide the maximum
+                        rate at which banners are grabbed. If this value
+                        is set higher than 1000, you should use 
+                        `ulimit -SSn 1000000` and `ulimit -SHn 1000000` to
+                        avoid running out of file descriptors (typically capped
+                        at 1024).
+
+-p, --port              The port which to connect to hosts on
+
+-t, --conn-timeout      Connection timeout (seconds). Give up on a host if connect
+                        has not completed by this time. Default: 4 seconds.
+
+-r, --read-timeout      Read timeout (seconds). Give up on a host if after
+                        connecting (and optionally sending data), it does
+                        not send any response by this time. Default: 4 seconds.
+
+-v, --verbosity         Set status verbosity. Status/error messages are outputed
+                        on stderr. This value can be 0-5, with 5 being the most
+                        verbose (LOG_TRACE). Default: 3 (LOG_INFO)
+
+-f, --format            Format to output banner responses. One of 'hex', 'ascii',
+                        or 'base64'. 
+                        'hex' outputs ascii hex characters, e.g. 48656c6c6f.  
+                        'ascii' outputs ascii, without separators, e.g. Hello
+                        'base64' outputs base64 encoding, e.g. SGVsbG8=
+                        Default is base64.
+
+-d, --data              Optional data file. This data will be sent to each host
+                        upon successful connection. Currently, this file does 
+                        not allow null characters, but supports up to 4
+                        occurances of the current host's IP address, by replacing
+                        %s with the string (inet_ntoa) of that host's IP address.   
+
+
+

examples/banner-grab/banner-grab-tcp.c

@@ -0,0 +1,440 @@
+/*
+ * Banner Grab Copyright 2013 Regents of the University of Michigan 
+ * 
+ * Licensed under the Apache License, Version 2.0 (the "License"); you may not
+ * use this file except in compliance with the License. You may obtain a copy
+ * of the License at http://www.apache.org/licenses/LICENSE-2.0
+ */
+
+#include <sys/socket.h>
+#include <netinet/in.h>
+#include <arpa/inet.h>
+#include "logger.h"
+
+#include <event.h>
+#include <event2/bufferevent_ssl.h>
+
+#include <getopt.h>
+#include <stdio.h>
+#include <stdlib.h>
+#include <string.h>
+#include <errno.h>
+#include <ulimit.h>
+
+#define MAX_BANNER_LEN 1024
+#define BASE64_ALPHABET  "ABCDEFGHIJKLMNOPQRSTUVWXYZabcdefghijklmnopqrstuvwxyz0123456789+/"
+
+struct config {
+	uint16_t port;
+	int connect_timeout;	// how long to wait for connecting (seconds)
+	int read_timeout;		// how long to wait once connected for the banner (seconds)
+	int current_running;
+	int max_concurrent;
+	struct event_base *base;
+	struct bufferevent *stdin_bev;
+	int stdin_closed;
+	enum {FORMAT_HEX, FORMAT_BASE64, FORMAT_ASCII} format;
+	char *send_str;
+	long send_str_size;
+
+	struct stats_st {
+		int init_connected_hosts;	// Number of hosts we have even tried to connect to
+		int connected_hosts;		// # hosts that picked up
+		int conn_timed_out;			// # hosts that timed out during connection
+		int read_timed_out;			// # hosts that connected, but sent no data (banner)
+		int timed_out;				// # hosts that timed out at all (conn_timed_out+read_timed_out)?
+		int completed_hosts;		// # hosts that presented a banner
+	} stats;
+};
+
+
+struct state {
+	struct config *conf;
+	uint32_t ip;
+	enum {CONNECTING, CONNECTED, RECEIVED} state;
+};
+
+void stdin_readcb(struct bufferevent *bev, void *arg);
+
+void print_status(evutil_socket_t fd, short events, void *arg)
+{
+	struct event *ev;
+	struct config *conf = arg;
+	struct event_base *base = conf->base;
+	struct timeval status_timeout = {1, 0};
+	ev = evtimer_new(base, print_status, conf);
+	evtimer_add(ev, &status_timeout);
+	(void)fd; (void)events;
+
+	log_info("banner-grab-tcp", "(%d/%d in use) - Totals: %d inited, %d connected, %d conn timeout, %d read timeout %d completed", 
+			conf->current_running, conf->max_concurrent, 
+			conf->stats.init_connected_hosts,
+			conf->stats.connected_hosts, conf->stats.conn_timed_out, 
+			conf->stats.read_timed_out, conf->stats.completed_hosts);
+}
+
+void decrement_cur_running(struct state *st)
+{	
+	struct config *conf = st->conf;
+	conf->current_running--;
+	log_debug("banner-grab-tcp", "done, down to %d",
+			conf->current_running);
+	if (evbuffer_get_length(bufferevent_get_input(conf->stdin_bev)) > 0) {
+		stdin_readcb(conf->stdin_bev, conf);
+	}
+	free(st);
+
+	if (conf->stdin_closed && conf->current_running == 0) {
+		// Done
+		log_info("banner-grab-tcp", "done");
+		print_status(0, 0, conf);
+		exit(0);
+	}
+
+}
+
+void connect_cb(struct bufferevent *bev, short events, void *arg)
+{
+	struct state *st = arg;
+	struct config *conf = st->conf;
+	struct in_addr addr;
+	addr.s_addr = st->ip;
+	if (events & BEV_EVENT_CONNECTED) {
+		struct timeval tv = {st->conf->read_timeout, 0};
+		log_debug("banner-grab-tcp", "%s connected", inet_ntoa(addr));
+	
+		// Send data
+		if (conf->send_str) {
+			struct evbuffer *evout = bufferevent_get_output(bev);
+			// HACK!!! TODO: make some messy parser that replaces ${IP} with IP etc
+			// and allow null characters
+			evbuffer_add_printf(evout, conf->send_str,
+					inet_ntoa(addr), inet_ntoa(addr), inet_ntoa(addr), inet_ntoa(addr));	
+		}
+
+		// Change from connect timeout to read timeout
+		bufferevent_set_timeouts(bev, &tv, &tv);
+
+		// Update state/stats
+		st->state = CONNECTED;
+		st->conf->stats.connected_hosts++;
+	} else {
+		if (st->state == CONNECTED) {
+			// Print out that we just didn't receive data
+			printf("%s X\n", inet_ntoa(addr));
+			fflush(stdout);
+
+			st->conf->stats.read_timed_out++;
+		} else {
+			st->conf->stats.conn_timed_out++;
+		}
+		log_debug("banner-grab-tcp", "%s bailing..", inet_ntoa(addr));
+		bufferevent_free(bev);
+		st->conf->stats.timed_out++;
+		decrement_cur_running(st);
+	}
+}
+
+// Grab these bytes, and close the connection.
+// Even if we don't need to read any bytes,
+// we have to have this so that libevent thinks we have
+// a read event, so that it can timeout TCP connects
+// (as a read timeout)
+void read_cb(struct bufferevent *bev, void *arg)
+{
+	struct evbuffer *in = bufferevent_get_input(bev);
+	struct state *st = arg;
+	size_t len = evbuffer_get_length(in);
+	struct in_addr addr;
+	addr.s_addr = st->ip;
+
+	log_debug("banner-grab-tcp", "read_cb for %s", inet_ntoa(addr));
+
+	if (len > MAX_BANNER_LEN) {
+		len = MAX_BANNER_LEN;
+	}
+
+	if (len > 0) {
+		// Grab the banner	
+		unsigned int i;
+		unsigned char *buf = malloc(len+1);
+
+		st->state = RECEIVED;
+
+		if (!buf) {
+			log_fatal("banner-grab-tcp", "cannot alloc %d byte buf", len+1);
+			return;
+		}
+		evbuffer_remove(in, buf, len);
+		
+		printf("%s ", inet_ntoa(addr));
+
+		if (st->conf->format == FORMAT_ASCII) {
+			// Ascii
+			buf[len] = '\0';
+			printf("%s\n", buf);
+		} else if (st->conf->format == FORMAT_HEX) {
+			// Hex output
+			for (i=0; i<len; i++) {
+				printf("%02x", buf[i]);
+			}
+			printf("\n");
+		} else if (st->conf->format == FORMAT_BASE64) {
+			// Base64
+			int i=0;
+			char out[4] = {0,0,0,0};
+			while (i < len) {
+				uint32_t value = 0;
+				value += (i < len) ? buf[i++] << 16 : 0; 	
+				value += (i < len) ? buf[i++] <<  8 : 0;
+				value += (i < len) ? buf[i++]       : 0;
+				out[0] = BASE64_ALPHABET[(value >> 18) & 0x3F];
+				out[1] = BASE64_ALPHABET[(value >> 12) & 0x3F];                                                                           
+				out[2] = BASE64_ALPHABET[(value >>  6) & 0x3F];
+				out[3] = BASE64_ALPHABET[(value      ) & 0x3F];
+				if (i < len) {
+					printf("%c%c%c%c", out[0], out[1], out[2], out[3]);
+				}
+			}
+			if (len > 0) {
+				switch (len % 3) {
+				case 1:
+					out[2] = '=';
+				case 2:
+					out[3] = '=';
+				default:
+					break;
+				}
+				printf("%c%c%c%c\n", out[0], out[1], out[2], out[3]);
+			}
+		}
+		fflush(stdout);
+
+		free(buf);
+		st->conf->stats.completed_hosts++;
+	}
+	bufferevent_free(bev);
+	decrement_cur_running(st);
+}
+
+void grab_banner(struct state *st)
+{
+	struct sockaddr_in addr;
+	struct bufferevent *bev;
+	struct timeval read_to = {st->conf->connect_timeout, 0};
+
+	addr.sin_family = AF_INET;
+	addr.sin_port = htons(st->conf->port);
+	addr.sin_addr.s_addr = st->ip;
+	log_debug("banner-grab-tcp", "connect %s:%d", 
+			  inet_ntoa(addr.sin_addr), st->conf->port);
+
+	bev = bufferevent_socket_new(st->conf->base, -1, BEV_OPT_CLOSE_ON_FREE);
+
+	bufferevent_set_timeouts(bev, &read_to, &read_to);
+
+	bufferevent_setcb(bev, read_cb, NULL, connect_cb, st);
+	bufferevent_enable(bev, EV_READ);
+
+	st->state = CONNECTING;
+
+	st->conf->stats.init_connected_hosts++;
+
+	if (bufferevent_socket_connect(bev, 
+		(struct sockaddr *)&addr, sizeof(addr)) < 0) {
+		log_warn("banner-grab-tcp", "could not connect socket %d (%d open) %d %d", 
+			bufferevent_getfd(bev), st->conf->current_running, errno, ENFILE);
+		perror("connect");
+
+		
+		bufferevent_free(bev);
+		decrement_cur_running(st);
+		return;
+	}
+}
+
+void stdin_eventcb(struct bufferevent *bev, short events, void *ptr) {
+	struct config *conf = ptr;
+
+	if (events & BEV_EVENT_EOF) {
+		log_debug("banner-grab-tcp", 
+				  "received EOF; quitting after buffer empties");
+		conf->stdin_closed = 1; 
+		if (conf->current_running == 0) {
+			log_info("banner-grab-tcp", "done");
+			print_status(0, 0, conf);
+			exit(0);
+		}
+	} 
+}
+
+void stdin_readcb(struct bufferevent *bev, void *arg)
+{
+	struct evbuffer *in = bufferevent_get_input(bev);
+	struct config *conf = arg;
+
+	log_debug("banner-grab-tcp", "stdin cb %d < %d ?", 
+		conf->current_running, conf->max_concurrent);
+
+	while (conf->current_running < conf->max_concurrent && 
+		   evbuffer_get_length(in) > 0) {
+		char *ip_str;
+		size_t line_len;
+		char *line = evbuffer_readln(in, &line_len, EVBUFFER_EOL_LF);
+		struct state *st;
+		if (!line)
+			break;
+		log_debug("banner-grab-tcp", "line: %s", line);
+
+		ip_str = line;
+		/*
+		port_str = strstr(line, ":") + 1;
+		if (!port_str)
+			port_str = strstr(line, " ") + 1;
+		if (!port_str)
+			break;
+
+		*(port_str-1) = '\0';
+		port = atoi(port_str);
+		*/
+		//printf("scanning %s:%d\n", ip
+		
+		conf->current_running++;
+		st = malloc(sizeof(*st));
+		st->conf = conf;
+		st->ip = inet_addr(ip_str);
+		grab_banner(st);
+	}
+}
+
+int main(int argc, char *argv[])
+{
+	struct event_base *base;
+	struct event *status_timer;
+	struct timeval status_timeout = {1, 0};
+	int c;
+	struct option long_options[] = {
+		{"concurrent", required_argument, 0, 'c'},
+		{"port", required_argument, 0, 'p'},
+		{"conn-timeout", required_argument, 0, 't'},
+		{"read-timeout", required_argument, 0, 'r'},
+		{"verbosity", required_argument, 0, 'v'},
+		{"format", no_argument, 0, 'f'},
+		{"data", required_argument, 0, 'd'},
+		{0, 0, 0, 0} };
+
+	struct config conf;
+	int ret;
+	FILE *fp;
+
+	log_init(stderr, LOG_INFO);
+
+	ret = ulimit(4, 1000000);	// Allow us to open 1 million fds (instead of 1024)
+	if (ret < 0) {
+		log_fatal("banner-grab-tcp", "cannot set ulimit");
+		perror("ulimit");
+		exit(1);
+	}
+
+	base = event_base_new();
+	conf.base = base;
+
+	// buffer stdin as an event
+	conf.stdin_bev = bufferevent_socket_new(base, 0, BEV_OPT_DEFER_CALLBACKS);
+	bufferevent_setcb(conf.stdin_bev, stdin_readcb, NULL, stdin_eventcb, &conf);
+	bufferevent_enable(conf.stdin_bev, EV_READ);
+
+	// Status timer
+	status_timer = evtimer_new(base, print_status, &conf);
+	evtimer_add(status_timer, &status_timeout);
+
+	// Defaults
+	conf.max_concurrent = 1;
+	conf.current_running = 0;
+	memset(&conf.stats, 0, sizeof(conf.stats));
+	conf.connect_timeout = 4;
+	conf.read_timeout = 4;
+	conf.stdin_closed = 0;
+	conf.format = FORMAT_BASE64;
+	conf.send_str = NULL;
+
+	// Parse command line args
+	while (1) {
+		int option_index = 0;
+		c = getopt_long(argc, argv, "c:p:t:r:v:f:d:",
+				long_options, &option_index);
+
+		if (c < 0) {
+			break;
+		}
+
+		switch (c) {
+		case 'c':
+			conf.max_concurrent = atoi(optarg);
+			break;
+		case 'p':
+			conf.port = atoi(optarg);
+			break;
+		case 't':
+			conf.connect_timeout = atoi(optarg);
+			break;
+		case 'r':
+			conf.read_timeout = atoi(optarg);
+			break;
+		case 'v':
+			if (atoi(optarg) >= 0 && atoi(optarg) <= 5) {
+				log_init(stderr, atoi(optarg));
+			}
+			break;
+		case 'f':
+			if (strcmp(optarg, "hex") == 0) {
+				conf.format = FORMAT_HEX;
+			} else if (strcmp(optarg, "base64") == 0) {
+				conf.format = FORMAT_BASE64;
+			} else if (strcmp(optarg, "ascii") == 0) {
+				conf.format = FORMAT_ASCII;
+			} else {
+				log_fatal("banner-grab-tcp", "Unknown format '%s'; use 'hex', 'base64', or 'ascii'",
+						  optarg);
+			}
+			break;
+		case 'd':
+			fp = fopen(optarg, "r");
+			if (!fp) {
+				log_error("banner-grab-tcp", "Could not open send data file '%s':", optarg);
+				perror("fopen");
+				exit(-1);
+			}
+			fseek(fp, 0L, SEEK_END);
+			conf.send_str_size = ftell(fp);
+			fseek(fp, 0L, SEEK_SET);
+			//assert(conf.send_str_size < 10000);	// jumbo frames?
+			conf.send_str = malloc(conf.send_str_size+1);
+			if (!conf.send_str) {
+				log_fatal("banner-grab-tcp", "Could not malloc %d bytes", conf.send_str_size+1);
+			}
+			if (fread(conf.send_str, conf.send_str_size, 1, fp) != 1) {
+				log_fatal("banner-grab-tcp", "Couldn't read from send data file '%s':", optarg);
+			}
+			conf.send_str[conf.send_str_size] = '\0';
+			fclose(fp);	
+			break;
+		case '?':
+			printf("Usage:\n");
+			printf("\t%s [-c max_concurrency] [-t connect_timeout] [-r read_timeout] \n\t"
+				   "[-v verbosity=0-5] [-d send_data_file] [-f ascii|hex|base64] -p port\n", argv[0]);
+			exit(1);
+		default:
+			log_info("banner-grab-tcp", "hmmm..");
+			break;
+		}
+	}
+	
+	log_info("banner-grab-tcp", "Using port %d with max_concurrency %d, %d s conn timeout, %d s read timeout",
+			conf.port, conf.max_concurrent, conf.connect_timeout, conf.read_timeout);
+
+	event_base_dispatch(base);
+
+	return 0;
+}
+

examples/banner-grab/http-req

@@ -0,0 +1,3 @@
+GET / HTTP/1.1
+Host: %s
+

examples/forge-socket/Makefile

@@ -0,0 +1,22 @@
+
+
+CFLAGS+=-I../../lib/ -I../../forge_socket -Wall
+LDFLAGS+=-lpcap -levent -levent_extra -lm
+VPATH=../../lib/
+
+# from dpkg-buildflags --get CFLAGS, but use stack-protector-all and fPIC
+GCCHARDENING=-g -O2 -fstack-protector-all --param=ssp-buffer-size=4 -Wformat -Wformat-security -Werror=format-security -fPIC
+# from gpkg-buildflags --get LDFLAGS, + -z,now
+LDHARDENING=-Wl,-Bsymbolic-functions -Wl,-z,relro,-z,now
+
+CFLAGS+=$(GCCHARDENING)
+LDFLAGS+=$(LDHARDENING)
+
+
+all: forge-socket
+
+forge-socket: forge-socket.o logger.o
+	$(CC) $(CFLAGS) $^ -o $@ $(LDFLAGS)
+
+clean:
+	rm -f forge-socket *.o

examples/forge-socket/README

@@ -0,0 +1,73 @@
+Forge-socket banner grab
+======
+
+This utility, in combination with a kernel module
+(https://github.com/ewust/forge_socket/) will complete the half-open connection
+created by ZMap during a TCP-scan, optionally send a small message, and wait
+for the hosts response. The response is then printed along with their IP
+address on stdout. Periodic status messages appear on stderr.
+
+This utility is functionally equivalent to banner-grab-tcp, however, instead of
+having the kernel send a RST packet for the server's SYN+ACK, and
+banner-grab-tcp attempting to start a fresh TCP connection with the host,
+forge-socket will take the parameters of the SYN+ACK packet, and use a kernel
+module to add it as an ESTABLISHED TCP connection socket. Then, the
+forge-socket user-space program can use this socket to send() and recv() as
+normal, and completes the banner-grab process (optionally send a small message,
+and receive the server's response).
+
+
+
+USING:
+-----
+# Install forge-socket to the ZMap root directory:
+cd ./zmap/
+git clone git@github.com:ewust/forge_socket.git
+cd forge_socket
+make
+sudo insmod forge_socket.ko
+
+# Don't send RST packets (forge-socket will complete these connections instead)
+sudo iptables -A OUTPUT -p tcp -m tcp --tcp-flags RST,RST RST,RST -j DROP
+
+# Use ZMap + forge-socket simultaneously:
+make
+#echo -e -n "GET / HTTP/1.1\r\nHost: %s\r\n\r\n" > http-req
+sudo su
+ulimit -SHn 1000000 && ulimit -SSn 1000000
+zmap -p 80 -B 50M -N 1000 -O extended_file -o - | ./forge-socket -c 8000 -d http-req > http-banners.out
+
+
+The options are similar to banner-grab-tcp, except there is no connection timeout :)
+
+OPTIONS:
+-----
+-c, --concurent         Number of connections that can be going on at once.
+                        This, combined with timeouts, will decide the maximum
+                        rate at which banners are grabbed. If this value
+                        is set higher than 1000, you should use 
+                        `ulimit -SSn 1000000` and `ulimit -SHn 1000000` to
+                        avoid running out of file descriptors (typically capped
+                        at 1024).
+
+-r, --read-timeout      Read timeout (seconds). Give up on a host if after
+                        connecting (and optionally sending data), it does
+                        not send any response by this time. Default: 4 seconds.
+
+-v, --verbosity         Set status verbosity. Status/error messages are outputed
+                        on stderr. This value can be 0-5, with 5 being the most
+                        verbose (LOG_TRACE). Default: 3 (LOG_INFO)
+
+-f, --format            Format to output banner responses. One of 'hex', 'ascii',
+                        or 'base64'. 
+                        'hex' outputs ascii hex characters, e.g. 48656c6c6f.  
+                        'ascii' outputs ascii, without separators, e.g. Hello
+                        'base64' outputs base64 encoding, e.g. SGVsbG8=
+                        Default is base64.
+
+-d, --data              Optional data file. This data will be sent to each host
+                        upon successful connection. Currently, this file does 
+                        not allow null characters, but supports up to 4
+                        occurances of the current host's IP address, by replacing
+                        %s with the string (inet_ntoa) of that host's IP address.   
+

examples/forge-socket/forge-socket.c

@@ -0,0 +1,496 @@
+/*
+ * Forge Socket Banner Grab Copyright 2013 Regents of the University of Michigan 
+ * 
+ * Licensed under the Apache License, Version 2.0 (the "License"); you may not
+ * use this file except in compliance with the License. You may obtain a copy
+ * of the License at http://www.apache.org/licenses/LICENSE-2.0
+ */
+
+#include <sys/socket.h>
+#include <netinet/in.h>
+#include <arpa/inet.h>
+#include "logger.h"
+
+#include <event.h>
+#include <event2/bufferevent_ssl.h>
+
+#include <getopt.h>
+#include <stdio.h>
+#include <stdlib.h>
+#include <string.h>
+#include <errno.h>
+#include <ulimit.h>
+
+#include "forge_socket.h"
+
+#define MAX_BANNER_LEN 1024
+#define BASE64_ALPHABET  "ABCDEFGHIJKLMNOPQRSTUVWXYZabcdefghijklmnopqrstuvwxyz0123456789+/"
+
+struct config {
+	int read_timeout;		// how long to wait once connected for the banner (seconds)
+	int current_running;
+	int max_concurrent;
+	struct event_base *base;
+	struct bufferevent *stdin_bev;
+	int stdin_closed;
+	enum {FORMAT_HEX, FORMAT_BASE64, FORMAT_ASCII} format;
+	char *send_str;
+	long send_str_size;
+
+	struct stats_st {
+		int init_connected_hosts;	// Number of hosts we have even tried to connect to
+		int connected_hosts;		// # hosts that picked up
+		int conn_timed_out;			// # hosts that timed out during connection
+		int read_timed_out;			// # hosts that connected, but sent no data (banner)
+		int timed_out;				// # hosts that timed out at all (conn_timed_out+read_timed_out)?
+		int completed_hosts;		// # hosts that presented a banner
+	} stats;
+};
+
+
+struct state {
+	struct config *conf;
+    uint32_t src_ip;
+    uint32_t dst_ip;
+    uint16_t sport;
+    uint16_t dport;
+    uint32_t seq;
+    uint32_t seq_ack;
+	enum {CONNECTING, CONNECTED, RECEIVED} state;
+};
+
+void stdin_readcb(struct bufferevent *bev, void *arg);
+
+void print_status(evutil_socket_t fd, short events, void *arg)
+{
+	struct event *ev;
+	struct config *conf = arg;
+	struct event_base *base = conf->base;
+	struct timeval status_timeout = {1, 0};
+	ev = evtimer_new(base, print_status, conf);
+	evtimer_add(ev, &status_timeout);
+	(void)fd; (void)events;
+
+	log_info("forge-socket", "(%d/%d in use) - Totals: %d inited, %d connected, %d conn timeout, %d read timeout %d completed", 
+			conf->current_running, conf->max_concurrent, 
+			conf->stats.init_connected_hosts,
+			conf->stats.connected_hosts, conf->stats.conn_timed_out, 
+			conf->stats.read_timed_out, conf->stats.completed_hosts);
+}
+
+void decrement_cur_running(struct state *st)
+{	
+	struct config *conf = st->conf;
+	conf->current_running--;
+	log_debug("forge-socket", "done, down to %d",
+			conf->current_running);
+	if (evbuffer_get_length(bufferevent_get_input(conf->stdin_bev)) > 0) {
+		stdin_readcb(conf->stdin_bev, conf);
+	}
+	free(st);
+
+	if (conf->stdin_closed && conf->current_running == 0) {
+		// Done
+		log_info("forge-socket", "done");
+		print_status(0, 0, conf);
+		exit(0);
+	}
+
+}
+
+void event_cb(struct bufferevent *bev, short events, void *arg)
+{
+	struct state *st = arg;
+	struct config *conf = st->conf;
+	struct in_addr addr;
+	addr.s_addr = st->src_ip;
+	if (events & BEV_EVENT_CONNECTED) {
+		log_error("forge-socket", "%s connected - wat?", inet_ntoa(addr));
+	
+	} else {
+		if (st->state == CONNECTED) {
+			// Print out that we just didn't receive data
+			printf("%s X\n", inet_ntoa(addr));
+			fflush(stdout);
+
+			conf->stats.read_timed_out++;
+		} else {
+			conf->stats.conn_timed_out++;
+		}
+		log_debug("forge-socket", "%s bailing..", inet_ntoa(addr));
+		bufferevent_free(bev);
+		conf->stats.timed_out++;
+		decrement_cur_running(st);
+	}
+}
+
+// Grab these bytes, and close the connection.
+// Even if we don't need to read any bytes,
+// we have to have this so that libevent thinks we have
+// a read event, so that it can timeout TCP connects
+// (as a read timeout)
+void read_cb(struct bufferevent *bev, void *arg)
+{
+	struct evbuffer *in = bufferevent_get_input(bev);
+	struct state *st = arg;
+	size_t len = evbuffer_get_length(in);
+	struct in_addr addr;
+	addr.s_addr = st->src_ip;
+
+	log_debug("forge-socket", "read_cb for %s", inet_ntoa(addr));
+
+	if (len > MAX_BANNER_LEN) {
+		len = MAX_BANNER_LEN;
+	}
+
+	if (len > 0) {
+		// Grab the banner	
+		unsigned int i;
+		unsigned char *buf = malloc(len+1);
+
+		st->state = RECEIVED;
+
+		if (!buf) {
+			log_fatal("forge-socket", "cannot alloc %d byte buf", len+1);
+			return;
+		}
+		evbuffer_remove(in, buf, len);
+		
+		printf("%s ", inet_ntoa(addr));
+
+		if (st->conf->format == FORMAT_ASCII) {
+			// Ascii
+			buf[len] = '\0';
+			printf("%s\n", buf);
+		} else if (st->conf->format == FORMAT_HEX) {
+			// Hex output
+			for (i=0; i<len; i++) {
+				printf("%02x", buf[i]);
+			}
+			printf("\n");
+		} else if (st->conf->format == FORMAT_BASE64) {
+			// Base64
+			int i=0;
+			char out[4] = {0,0,0,0};
+			while (i < len) {
+				uint32_t value = 0;
+				value += (i < len) ? buf[i++] << 16 : 0; 	
+				value += (i < len) ? buf[i++] <<  8 : 0;
+				value += (i < len) ? buf[i++]       : 0;
+				out[0] = BASE64_ALPHABET[(value >> 18) & 0x3F];
+				out[1] = BASE64_ALPHABET[(value >> 12) & 0x3F];                                                                           
+				out[2] = BASE64_ALPHABET[(value >>  6) & 0x3F];
+				out[3] = BASE64_ALPHABET[(value      ) & 0x3F];
+				if (i < len) {
+					printf("%c%c%c%c", out[0], out[1], out[2], out[3]);
+				}
+			}
+			if (len > 0) {
+				switch (len % 3) {
+				case 1:
+					out[2] = '=';
+				case 2:
+					out[3] = '=';
+				default:
+					break;
+				}
+				printf("%c%c%c%c\n", out[0], out[1], out[2], out[3]);
+			}
+		}
+		fflush(stdout);
+
+		free(buf);
+		st->conf->stats.completed_hosts++;
+	}
+	bufferevent_free(bev);
+	decrement_cur_running(st);
+}
+
+int set_sock_state(int sock, struct tcp_state *st)
+{
+    struct sockaddr_in sin;
+    sin.sin_family      = AF_INET;
+    sin.sin_addr.s_addr = st->src_ip;
+    sin.sin_port        = st->sport;
+
+    int value = 1;
+    if (setsockopt(sock, SOL_SOCKET, SO_REUSEADDR, &value, sizeof(value)) < 0) {
+        perror("setsockopt SO_REUSEADDR");
+        return -1;
+    }
+
+    if (setsockopt(sock, SOL_IP, IP_TRANSPARENT, &value, sizeof(value)) < 0) {
+        perror("setsockopt IP_TRANSPARENT");
+        return -1;
+    }
+
+    if (bind(sock, (struct sockaddr *)&sin, sizeof(sin)) < 0) {
+        perror("bind");
+        return -1;
+    }
+
+    if (setsockopt(sock, IPPROTO_TCP, TCP_STATE, st, sizeof(struct tcp_state)) < 0) {
+        perror("setsockopt TCP_STATE");
+        return -1;
+    }
+
+    return 0;
+}
+
+
+
+
+void grab_banner(struct state *st)
+{
+	struct sockaddr_in addr;
+	struct bufferevent *bev;
+	struct timeval read_to = {st->conf->read_timeout, 0};
+    struct tcp_state tcp_st;
+    int sock = socket(AF_INET, SOCK_FORGE, 0);
+
+    addr.sin_addr.s_addr = st->src_ip;
+
+    if (sock < 0) {
+        perror("SOCK_FORGE socket");
+        log_fatal("forge_socket", "(did you insmod forge_socket.ko?)");
+        return;
+    }
+
+    memset(&tcp_st, 0, sizeof(tcp_st));
+    
+    // These need to be in network order for forge socket"
+    tcp_st.src_ip = st->dst_ip;
+    tcp_st.dst_ip = st->src_ip;
+    tcp_st.sport = htons(st->dport);
+    tcp_st.dport = htons(st->sport);
+
+    // This should be in ???
+    tcp_st.seq = st->seq_ack;
+    tcp_st.ack = (st->seq + 1);
+
+    tcp_st.snd_wnd = 0x1000;
+    tcp_st.rcv_wnd = 0x1000;
+    
+    tcp_st.snd_una = tcp_st.seq;
+	st->state = CONNECTING;
+	st->conf->stats.init_connected_hosts++;
+
+    // consider this a non-blocking, but completed "connect()". heh.
+    if (set_sock_state(sock, &tcp_st) != 0) {
+        log_error("forge_socket", "set_sock_state failed\n");
+		decrement_cur_running(st);
+        return;
+    }
+
+    evutil_make_socket_nonblocking(sock);
+
+	bev = bufferevent_socket_new(st->conf->base, sock, BEV_OPT_CLOSE_ON_FREE);
+
+	bufferevent_set_timeouts(bev, &read_to, &read_to);
+
+	bufferevent_setcb(bev, read_cb, NULL, event_cb, st);
+	bufferevent_enable(bev, EV_READ);
+
+
+	// Send data
+	if (st->conf->send_str) {
+		struct evbuffer *evout = bufferevent_get_output(bev);
+		// HACK!!! TODO: make some messy parser that replaces ${IP} with IP etc
+		// and allow null characters
+		evbuffer_add_printf(evout, st->conf->send_str,
+				inet_ntoa(addr.sin_addr), inet_ntoa(addr.sin_addr),
+                inet_ntoa(addr.sin_addr), inet_ntoa(addr.sin_addr));	
+        log_trace("forge-socket", "sent str to %s", inet_ntoa(addr.sin_addr)); 
+	}
+
+	// Update state/stats
+	st->state = CONNECTED;
+	st->conf->stats.connected_hosts++;
+
+    log_trace("forge-socket", "go %s go! read a byte!!", inet_ntoa(addr.sin_addr));
+}
+
+void stdin_eventcb(struct bufferevent *bev, short events, void *ptr) {
+	struct config *conf = ptr;
+
+	if (events & BEV_EVENT_EOF) {
+		log_debug("forge-socket", 
+				  "received EOF; quitting after buffer empties");
+		conf->stdin_closed = 1; 
+		if (conf->current_running == 0) {
+			log_info("forge-socket", "done");
+			print_status(0, 0, conf);
+			exit(0);
+		}
+	} 
+}
+
+void stdin_readcb(struct bufferevent *bev, void *arg)
+{
+	struct evbuffer *in = bufferevent_get_input(bev);
+	struct config *conf = arg;
+
+	log_debug("forge-socket", "stdin cb %d < %d ?", 
+		conf->current_running, conf->max_concurrent);
+
+	while (conf->current_running < conf->max_concurrent && 
+		   evbuffer_get_length(in) > 0) {
+		size_t line_len;
+		char *line = evbuffer_readln(in, &line_len, EVBUFFER_EOL_LF);
+		struct state *st;
+		if (!line)
+			break;
+		log_debug("forge-socket", "line: '%s'", line);
+
+        //synack, 77.176.116.205, 141.212.121.125, 443, 49588, 3628826326, 3441755636, 0, 0,2013-08-11 19:16:05.799
+        char synack[12];
+        char srcip[INET_ADDRSTRLEN], dstip[INET_ADDRSTRLEN];
+        uint32_t seq, seq_ack;
+        uint16_t sport, dport;
+        int cooldown, repeat=1;
+
+
+        int ret = sscanf(line, "%11[^,], %15[^,], %15[^,], %hu, %hu, %u, %u, %d, %d,%*s",
+            synack, srcip, dstip, &sport, &dport, &seq, &seq_ack, &cooldown, &repeat);
+
+        log_trace("forge-socket", "%d '%s' sip: '%s', dip: '%s', sport: %d, dport: %d, seq: %d, seq_ack: %d",
+            ret, synack, srcip, dstip, sport, dport, seq, seq_ack);
+
+        if (ret==9 && !repeat && strcmp(synack, "synack") == 0) {
+		    st = malloc(sizeof(*st));
+		    st->conf = conf;
+		    st->src_ip = inet_addr(srcip);
+            st->dst_ip = inet_addr(dstip);
+            st->sport = sport;
+            st->dport = dport;
+            st->seq = seq;
+            st->seq_ack = seq_ack;
+		
+		    conf->current_running++;
+		    grab_banner(st);
+        }
+	}
+}
+
+int main(int argc, char *argv[])
+{
+	struct event_base *base;
+	struct event *status_timer;
+	struct timeval status_timeout = {1, 0};
+	int c;
+	struct option long_options[] = {
+		{"concurrent", required_argument, 0, 'c'},
+		{"read-timeout", required_argument, 0, 'r'},
+		{"verbosity", required_argument, 0, 'v'},
+		{"format", no_argument, 0, 'f'},
+		{"data", required_argument, 0, 'd'},
+		{0, 0, 0, 0} };
+
+	struct config conf;
+	int ret;
+	FILE *fp;
+
+	log_init(stderr, LOG_INFO);
+
+	ret = ulimit(4, 1000000);	// Allow us to open 1 million fds (instead of 1024)
+	if (ret < 0) {
+		log_fatal("forge-socket", "cannot set ulimit");
+		perror("ulimit");
+		exit(1);
+	}
+
+	base = event_base_new();
+	conf.base = base;
+
+	// buffer stdin as an event
+	conf.stdin_bev = bufferevent_socket_new(base, 0, BEV_OPT_DEFER_CALLBACKS);
+	bufferevent_setcb(conf.stdin_bev, stdin_readcb, NULL, stdin_eventcb, &conf);
+	bufferevent_enable(conf.stdin_bev, EV_READ);
+
+	// Status timer
+	status_timer = evtimer_new(base, print_status, &conf);
+	evtimer_add(status_timer, &status_timeout);
+
+	// Defaults
+	conf.max_concurrent = 1;
+	conf.current_running = 0;
+	memset(&conf.stats, 0, sizeof(conf.stats));
+	conf.read_timeout = 4;
+	conf.stdin_closed = 0;
+	conf.format = FORMAT_BASE64;
+	conf.send_str = NULL;
+
+	// Parse command line args
+	while (1) {
+		int option_index = 0;
+		c = getopt_long(argc, argv, "c:t:r:v:f:d:",
+				long_options, &option_index);
+
+		if (c < 0) {
+			break;
+		}
+
+		switch (c) {
+		case 'c':
+			conf.max_concurrent = atoi(optarg);
+			break;
+		case 'r':
+			conf.read_timeout = atoi(optarg);
+			break;
+		case 'v':
+			if (atoi(optarg) >= 0 && atoi(optarg) <= 5) {
+				log_init(stderr, atoi(optarg));
+			}
+			break;
+		case 'f':
+			if (strcmp(optarg, "hex") == 0) {
+				conf.format = FORMAT_HEX;
+			} else if (strcmp(optarg, "base64") == 0) {
+				conf.format = FORMAT_BASE64;
+			} else if (strcmp(optarg, "ascii") == 0) {
+				conf.format = FORMAT_ASCII;
+			} else {
+				log_fatal("forge-socket", "Unknown format '%s'; use 'hex', 'base64', or 'ascii'",
+						  optarg);
+			}
+			break;
+		case 'd':
+			fp = fopen(optarg, "r");
+			if (!fp) {
+				log_error("forge-socket", "Could not open send data file '%s':", optarg);
+				perror("fopen");
+				exit(-1);
+			}
+			fseek(fp, 0L, SEEK_END);
+			conf.send_str_size = ftell(fp);
+			fseek(fp, 0L, SEEK_SET);
+			//assert(conf.send_str_size < 10000);	// jumbo frames?
+			conf.send_str = malloc(conf.send_str_size+1);
+			if (!conf.send_str) {
+				log_fatal("forge-socket", "Could not malloc %d bytes", conf.send_str_size+1);
+			}
+			if (fread(conf.send_str, conf.send_str_size, 1, fp) != 1) {
+				log_fatal("forge-socket", "Couldn't read from send data file '%s':", optarg);
+			}
+			conf.send_str[conf.send_str_size] = '\0';
+			fclose(fp);	
+			break;
+		case '?':
+			printf("Usage:\n");
+			printf("\t%s [-c max_concurrency] [-r read_timeout] \n\t"
+				   "[-v verbosity=0-5] [-d send_data_file] [-f ascii|hex|base64]\n", argv[0]);
+			exit(1);
+		default:
+			log_info("forge-socket", "hmmm..");
+			break;
+		}
+	}
+	
+	log_info("forge-socket", "Using max_concurrency %d, %d s read timeout",
+			conf.max_concurrent, conf.read_timeout);
+
+	event_base_dispatch(base);
+
+	return 0;
+}
+

lib/blacklist.c

@@ -0,0 +1,129 @@
+/*
+ * Blacklist Copyright 2013 Regents of the University of Michigan 
+ * 
+ * Licensed under the Apache License, Version 2.0 (the "License"); you may not
+ * use this file except in compliance with the License. You may obtain a copy
+ * of the License at http://www.apache.org/licenses/LICENSE-2.0
+ */
+ 
+#include "blacklist.h"
+
+#include <errno.h>
+#include <stdio.h>
+#include <assert.h>
+#include <stdint.h>
+#include <stdlib.h>
+#include <string.h>
+#include <sys/socket.h>
+#include <netinet/in.h>
+#include <arpa/inet.h>
+#include "constraint.h"
+#include "logger.h"
+
+#define ADDR_DISALLOWED 0
+#define ADDR_ALLOWED 1
+
+static constraint_t *constraint = NULL;
+
+// check whether a single IP address is allowed to be scanned.
+//		1 => is allowed
+//		0 => is not allowed
+int blacklist_is_allowed(uint32_t s_addr) {
+	return constraint_lookup_ip(constraint, ntohl(s_addr)) == ADDR_ALLOWED;
+}
+
+// blacklist a CIDR network allocation
+// e.g. blacklist_add("128.255.134.0", 24)
+void blacklist_prefix(char *ip, int prefix_len)
+{
+	assert(constraint);
+	constraint_set(constraint, ntohl(inet_addr(ip)), prefix_len, ADDR_DISALLOWED);
+}
+
+// whitelist a CIDR network allocation
+void whitelist_prefix(char *ip, int prefix_len)
+{
+	assert(constraint);
+	constraint_set(constraint, ntohl(inet_addr(ip)), prefix_len, ADDR_ALLOWED);
+}
+
+static int init(char *file, const char *name, int value)
+{
+	FILE *fp;
+	char line[1000];
+
+	fp = fopen(file, "r");
+	if (fp == NULL) {
+		log_fatal(name, "Unable to open %s file: %s: %s", name, file, strerror(errno));
+	}
+
+	while (fgets(line, sizeof(line), fp) != NULL) {
+		char *comment = strchr(line, '#');
+		if (comment) {
+			*comment = '\0';
+		}
+		char ip[33];
+		if ((sscanf(line, "%32s", ip)) == EOF) {
+			continue;
+		}
+		int prefix_len = 32;
+		char *slash = strchr(ip, '/');
+		if (slash) {  // split apart network and prefix length 
+			*slash = '\0';
+			char *end;
+			char *len = slash+1;
+			errno = 0;			
+			prefix_len = strtol(len, &end, 10);
+			if (end == len || errno != 0 || prefix_len < 0 || prefix_len > 32) {
+				log_fatal(name, "Unable to parse %s file: %s ('%s' is not a valid prefix length)",  name, file, len);
+			}
+		}
+		struct in_addr addr;
+		if (inet_aton(ip, &addr) == 0) {
+			log_fatal(name,  "Unable to parse %s file: %s ('%s' is not a valid IP address)",  name, file, ip);
+		}
+		constraint_set(constraint, ntohl(addr.s_addr), prefix_len, value);
+		log_trace(name, "%sing %s/%i",
+				  name, ip, prefix_len);
+	}
+	fclose(fp);
+
+	return 0;
+}
+
+uint64_t blacklist_count_allowed()
+{
+	assert(constraint);
+	return constraint_count_ips(constraint, ADDR_ALLOWED);
+}
+
+uint64_t blacklist_count_not_allowed()
+{
+	assert(constraint);
+	return constraint_count_ips(constraint, ADDR_DISALLOWED);
+}
+
+
+// Initialize address constraints from whitelist and blacklist files.
+// Either can be set to NULL to omit.
+int blacklist_init_from_files(char *whitelist_filename, char *blacklist_filename)
+{
+	assert(!constraint);
+	if (whitelist_filename) {
+		// using a whitelist, so default to allowing nothing
+		constraint = constraint_init(ADDR_DISALLOWED);
+		log_trace("whitelist", "blacklisting 0.0.0.0/0");
+		init(whitelist_filename, "whitelist", ADDR_ALLOWED);
+	} else {
+		// no whitelist, so default to allowing everything
+		constraint = constraint_init(ADDR_ALLOWED);
+	}
+	if (blacklist_filename) {
+		init(blacklist_filename, "blacklist", ADDR_DISALLOWED);
+	}
+	constraint_optimize(constraint);
+	uint64_t allowed = blacklist_count_allowed();
+	log_debug("blacklist", "%lu addresses allowed to be scanned (%0.0f%% of address space)", 
+			  allowed, allowed*100./(1L << 32));
+	return 0;
+}

lib/blacklist.h

@@ -0,0 +1,13 @@
+#include <stdint.h>
+
+#ifndef _BLACKLIST_H
+#define _BLACKLIST_H
+
+int blacklist_is_allowed(uint32_t s_addr);
+void blacklist_prefix(char *ip, int prefix_len);
+void whitelist_prefix(char *ip, int prefix_len);
+int blacklist_init_from_files(char *whitelist, char*blacklist);
+uint64_t blacklist_count_allowed();
+uint64_t blacklist_count_not_allowed();
+
+#endif

lib/constraint.c

@@ -0,0 +1,384 @@
+#include <stdio.h>
+#include <stdlib.h>
+#include <stdint.h>
+#include <string.h>
+#include <errno.h>
+#include <assert.h>
+#include <sys/socket.h>
+#include <netinet/in.h>
+#include <arpa/inet.h>
+
+#include "../lib/constraint.h"
+#include "../lib/logger.h"
+
+//
+// Efficient address-space constraints  (AH 7/2013)
+//
+// This module uses a tree-based representation to efficiently
+// manipulate and query constraints on the address space to be
+// scanned.  It provides a value for every IP address, and these
+// values are applied by setting them for network prefixes.  Order
+// matters: setting a value replaces any existing value for that
+// prefix or subsets of it.  We use this to implement network
+// whitelisting and blacklisting.
+//
+// Think of setting values in this structure like painting
+// subnets with different colors.  We can paint subnets black to
+// exclude them and white to allow them.  Only the top color shows.
+// This makes for potentially very powerful constraint specifications.
+//
+// Internally, this is implemented using a binary tree, where each
+// node corresponds to a network prefix.  (E.g., the root is
+// 0.0.0.0/0, and its children, if present, are 0.0.0.0/1 and
+// 128.0.0.0/1.)  Each leaf of the tree stores the value that applies
+// to every address within the leaf's portion of the prefix space.
+//
+// As an optimization, after all values are set, we look up the
+// value or subtree for every /16 prefix and cache them as an array.
+// This lets subsequent lookups bypass the bottom half of the tree.
+//
+
+/*
+ * Constraint Copyright 2013 Regents of the University of Michigan 
+ * 
+ * Licensed under the Apache License, Version 2.0 (the "License"); you may not
+ * use this file except in compliance with the License. You may obtain a copy
+ * of the License at http://www.apache.org/licenses/LICENSE-2.0
+ */
+
+typedef struct node {
+	struct node *l;
+	struct node *r;
+	value_t value;
+} node_t;
+
+// As an optimization, we precompute lookups for every prefix of this
+// length:
+#define RADIX_LENGTH 16
+
+struct _constraint {
+	node_t *root;   // root node of the tree
+	node_t **radix; // array of nodes for every RADIX_LENGTH prefix
+	int optimized;  // is radix populated and up-to-date?
+};
+
+// Tree operations respect the invariant that every node that isn't a
+// leaf has exactly two children.
+#define IS_LEAF(node) ((node)->l == NULL)
+
+// Allocate a new leaf with the given value
+static node_t* _create_leaf(value_t value)
+{
+	node_t *node = malloc(sizeof(node_t));
+	assert(node);
+	node->l = NULL;
+	node->r = NULL;
+	node->value = value;
+	return node;
+}
+
+// Free the subtree rooted at node.
+static void _destroy_subtree(node_t *node)
+{
+	if (node == NULL)
+		return;
+	_destroy_subtree(node->l);
+	_destroy_subtree(node->r);
+	free(node);
+}
+
+// Convert from an internal node to a leaf.
+static void _convert_to_leaf(node_t *node)
+{
+	assert(node);
+	assert(!IS_LEAF(node));
+	_destroy_subtree(node->l);
+	_destroy_subtree(node->r);
+	node->l = NULL;
+	node->r = NULL;
+}
+
+// Recursive function to set value for a given network prefix within
+// the tree.  (Note: prefix must be in host byte order.)
+static void _set_recurse(node_t *node, uint32_t prefix, int len, value_t value)
+{
+	assert(node);
+	assert(0 <= len && len <= 32);
+
+	if (len == 0) {
+		// We're at the end of the prefix; make this a leaf and set the value.
+		if (!IS_LEAF(node)) {
+			_convert_to_leaf(node);
+		}
+		node->value = value;
+		return;
+	}
+
+	if (IS_LEAF(node)) {
+		// We're not at the end of the prefix, but we hit a leaf.
+		if (node->value == value) {
+			// A larger prefix has the same value, so we're done.
+			return;
+		}
+		// The larger prefix has a different value, so we need to convert it
+		// into an internal node and continue processing on one of the leaves.
+		node->l = _create_leaf(node->value);
+		node->r = _create_leaf(node->value);
+	}
+
+	// We're not at the end of the prefix, and we're at an internal
+	// node.  Recurse on the left or right subtree.
+	if (prefix & 0x80000000) {
+		_set_recurse(node->r, prefix << 1, len - 1, value);
+	} else {
+		_set_recurse(node->l, prefix << 1, len - 1, value);
+	}
+
+	// At this point, we're an internal node, and the value is set
+	// by one of our children or its descendent.  If both children are
+	// leaves with the same value, we can discard them and become a left.
+	if (IS_LEAF(node->r) && IS_LEAF(node->l) && node->r->value == node->l->value) {
+		node->value = node->l->value;
+		_convert_to_leaf(node);
+	}
+}
+
+// Set the value for a given network prefix, overwriting any existing
+// values on that prefix or subsets of it.
+// (Note: prefix must be in host byte order.)
+void constraint_set(constraint_t *con, uint32_t prefix, int len, value_t value)
+{
+	assert(con);
+	_set_recurse(con->root, prefix, len, value);
+	con->optimized = 0;
+}
+
+// Return the value pertaining to an address, according to the tree
+// starting at given root.  (Note: address must be in host byte order.)
+static int _lookup_ip(node_t *root, uint32_t address)
+{
+	assert(root);
+	node_t *node = root;
+	uint32_t mask = 0x80000000;
+	for (;;) {
+		if (IS_LEAF(node)) {
+			return node->value;
+		}
+		if (address & mask) {
+			node = node->r;
+		} else {
+			node = node->l;
+		}
+		mask >>= 1;
+	}
+}
+
+
+// Return the value pertaining to an address.
+// (Note: address must be in host byte order.)
+int constraint_lookup_ip(constraint_t *con, uint32_t address)
+{
+	assert(con);
+	if (con->optimized) {
+		// Use radix optimization
+		node_t *node = con->radix[address >> (32 - RADIX_LENGTH)];
+		if (IS_LEAF(node)) {
+			return node->value;
+		}
+		return _lookup_ip(node, address << RADIX_LENGTH);
+	} else {
+		// Do a full lookup using the tree
+		log_trace("constraint", "Unoptimized lookup");
+		return _lookup_ip(con->root, address);
+	}
+}
+
+// Implement count_ips by recursing on halves of the tree.  Size represents
+// the number of addresses in a prefix at the current level of the tree.
+static uint64_t _count_ips_recurse(node_t *node, value_t value, uint64_t size)
+{
+	assert(node);
+	if (IS_LEAF(node)) {
+		if (node->value == value) {
+			return size;
+		} else {
+			return 0;
+		}
+	}
+	return _count_ips_recurse(node->l, value, size >> 1) +
+		_count_ips_recurse(node->r, value, size >> 1);
+}
+
+// Return the number of addresses that have a given value.
+uint64_t constraint_count_ips(constraint_t *con, value_t value)
+{
+	assert(con);
+	return _count_ips_recurse(con->root, value, (uint64_t)1 << 32);
+}
+
+// Initialize the tree.
+// All addresses will initally have the given value.
+constraint_t* constraint_init(value_t value)
+{
+	log_trace("constraint", "Initializing");
+	constraint_t* con = malloc(sizeof(constraint_t));
+	con->root = _create_leaf(value);
+	con->radix = calloc(sizeof(node_t *), 1 << RADIX_LENGTH);
+	assert(con->radix);
+	con->optimized = 0;
+	return con;
+}
+
+// Deinitialize and free the tree.
+void constraint_free(constraint_t *con)
+{
+	assert(con);
+	log_trace("constraint", "Cleaning up");
+	_destroy_subtree(con->root);
+	free(con->radix);
+	free(con);
+}
+
+// Return a node that determines the values for the addresses with
+// the given prefix.  This is either the internal node that
+// corresponds to the end of the prefix or a leaf node that
+// encompasses the prefix. (Note: prefix must be in host byte order.)
+static node_t* _lookup_node(node_t *root, uint32_t prefix, int len)
+{
+	assert(root);
+	assert(0 <= len && len <= 32);
+
+	node_t *node = root;
+	uint32_t mask = 0x80000000;
+
+	for (int i=0; i < len; i++) {
+		if (IS_LEAF(node)) {
+			return node;
+		}
+		if (prefix & mask) {
+			node = node->r;
+		} else {
+			node = node->l;
+		}
+		mask >>= 1;
+	}
+	return node;
+}
+
+// After values have been set, precompute prefix lookups.
+void constraint_optimize(constraint_t *con)
+{
+	assert(con);
+	if (con->optimized) {
+		return;
+	}
+	log_trace("constraint", "Optimizing constraints");
+	for (uint32_t i=0; i < (1 << RADIX_LENGTH); i++) {
+		uint32_t prefix = i << (32 - RADIX_LENGTH);
+		con->radix[i] = _lookup_node(con->root, prefix, RADIX_LENGTH);
+	}
+	con->optimized = 1;
+}
+
+
+/*
+int main(void)
+{
+	log_init(stderr, LOG_DEBUG);
+
+	constraint_t *con = constraint_init(0);
+	constraint_set(con, ntohl(inet_addr("128.128.0.0")), 1, 22);
+	constraint_set(con, ntohl(inet_addr("128.128.0.0")), 1, 1);
+	constraint_set(con, ntohl(inet_addr("128.0.0.0")), 1, 1);
+	constraint_set(con, ntohl(inet_addr("10.0.0.0")), 24, 1);
+	constraint_set(con, ntohl(inet_addr("10.0.0.0")), 24, 0);
+	constraint_set(con, ntohl(inet_addr("10.11.12.0")), 24, 1);
+	constraint_set(con, ntohl(inet_addr("141.212.0.0")), 16, 0);
+
+	for (int x=1; x < 2; x++) {
+		if (x == 1) {
+			constraint_optimize(con);
+		}
+
+		printf("count(0)=%ld\n", constraint_count_ips(con, 0));
+		printf("count(1)=%ld\n", constraint_count_ips(con, 1));
+		printf("%d\n", constraint_lookup_ip(con,ntohl(inet_addr("10.11.12.0"))));
+		assert(constraint_count_ips(con, 0) + constraint_count_ips(con, 1) == (uint64_t)1 << 32);
+
+		uint32_t i=0, count=0;
+		do {
+			if (constraint_lookup_ip(con, i))
+	count++;
+		} while (++i != 0);
+		printf("derived count(1)=%u\n", count);
+	}
+
+	constraint_free(con);
+}
+*/
+
+/*
+static int init(constraint_t *con, char *file, const char *name, value_t value)
+{
+	FILE *fp;
+	char line[1000];
+	int blocked = 0;
+
+	fp = fopen(file, "r");
+	if (fp == NULL) {
+		log_fatal(name, "Unable to open %s file: %s: %s", 
+				name, file, strerror(errno));
+	}
+
+	while (fgets(line, sizeof(line), fp) != NULL) {
+		char *comment = strchr(line, '#');
+		if (comment) {
+			*comment = '\0';
+		}
+		char ip[33];
+		if ((sscanf(line, "%32s", ip)) == EOF) {
+			continue;
+		}
+		int prefix_len;
+		char *slash = strchr(ip, '/');
+		if (slash == NULL) {
+			log_fatal(name,
+		"Unable to parse %s file: %s",
+		name, file);
+		}
+		// split apart network and prefix length 
+		*slash = '\0';
+		prefix_len = atoi(&slash[1]);
+		constraint_set(con, ntohl(inet_addr(ip)), prefix_len, value);
+
+		blocked++;
+	}
+	fclose(fp);
+	return 0;
+}
+
+
+
+void main()
+{
+	log_init(stderr, LOG_TRACE);
+
+	constraint_t *con = constraint_init(1);
+	init(con, "blacklist.prefixes", "blacklist", 0);
+	//constraint_optimize(con);
+
+	printf("count(0)=%lu\n", constraint_count_ips(con, 0));
+	printf("count(1)=%lu\n", constraint_count_ips(con, 1));
+
+	uint32_t i=0, count=0;
+	do {
+		if (constraint_lookup_ip(con, i))
+			count++;
+	} while (++i != 0);
+	printf("derived count(1)=%u\n", count);
+
+	constraint_free(con);
+
+}
+
+ */

lib/constraint.h

@@ -0,0 +1,14 @@
+#ifndef _CONSTRAINT_H
+#define _CONSTRAINT_H
+
+typedef struct _constraint constraint_t;
+typedef int value_t;
+
+constraint_t* constraint_init(value_t value);
+void constraint_free(constraint_t *con);
+void constraint_set(constraint_t *con, uint32_t prefix, int len, value_t value);
+void constraint_optimize(constraint_t *con);
+int constraint_lookup_ip(constraint_t *con, uint32_t address);
+uint64_t constraint_count_ips(constraint_t *con, value_t value);
+
+#endif //_CONSTRAINT_H

lib/logger.c

@@ -0,0 +1,126 @@
+/*
+ * Logger Copyright 2013 Regents of the University of Michigan 
+ * 
+ * Licensed under the Apache License, Version 2.0 (the "License"); you may not
+ * use this file except in compliance with the License. You may obtain a copy
+ * of the License at http://www.apache.org/licenses/LICENSE-2.0
+ */
+ 
+#include <stdlib.h>
+#include <stdio.h>
+#include <string.h>
+#include <assert.h>
+#include <sys/time.h>
+#include <time.h>
+#include <math.h>
+
+#include "logger.h"
+
+#ifndef HEADER_ZUTIL_LOGGER_H
+#define HEADER_ZUTIL_LOGGER_H
+
+static enum LogLevel log_output_level = LOG_INFO;
+
+static FILE *log_output_stream = NULL;
+
+static const char *log_level_name[] = { 
+	"FATAL", "ERROR", "WARN", "INFO", "DEBUG", "TRACE" };
+
+static int LogLogVA(enum LogLevel level, const char *loggerName,
+		const char *logMessage, va_list args)
+{
+	if (!log_output_stream) {
+		log_output_stream = stdout;
+	}
+	if (level <= log_output_level) {
+		const char *levelName = log_level_name[level];
+		assert(level < sizeof(log_level_name));
+
+		struct timeval now;
+		char timestamp[256];
+		gettimeofday(&now, NULL);
+		time_t sec = now.tv_sec;
+		struct tm* ptm = localtime(&sec);
+		strftime(timestamp, 20, "%b %d %H:%M:%S", ptm);
+		fprintf(log_output_stream, "%s.%03ld [%s] ", 
+				timestamp, now.tv_usec/1000, levelName);
+		if (loggerName) {
+			fprintf(log_output_stream, "%s: ", loggerName);
+		}
+		if (logMessage) {
+			vfprintf(log_output_stream, logMessage, args);
+		}
+		if (loggerName || logMessage) {
+			fputs("\n", log_output_stream);
+		}
+	}
+	return 0;
+}
+
+int log_fatal(const char *name, const char *message, ...) {
+	va_list va; va_start(va, message);
+	LogLogVA(LOG_FATAL, name, message, va);
+	va_end(va);
+	exit(EXIT_FAILURE);
+}
+
+int log_error(const char *name, const char *message, ...) {
+	va_list va; va_start(va, message);
+	int ret = LogLogVA(LOG_ERROR, name, message, va);
+	va_end(va);
+	return ret;
+}
+
+int log_warn(const char *name, const char *message, ...) {
+	va_list va; va_start(va, message);
+	int ret = LogLogVA(LOG_WARN, name, message, va);
+	va_end(va);
+	return ret;
+}
+
+int log_info(const char *name, const char *message, ...) {
+	va_list va; va_start(va, message);
+	int ret = LogLogVA(LOG_INFO, name, message, va);
+	va_end(va);
+	return ret;
+}
+
+int log_debug(const char *name, const char *message, ...) {
+	va_list va; va_start(va, message);
+	int ret = LogLogVA(LOG_DEBUG, name, message, va);
+	va_end(va);
+	return ret;
+}
+
+extern int log_trace(const char *name, const char *message, ...) {
+	va_list va; va_start(va, message);
+	int ret = LogLogVA(LOG_TRACE, name, message, va);
+	va_end(va);
+	return ret;
+}
+
+int log_init(FILE *stream, enum LogLevel level)
+{
+	log_output_stream = stream;
+	log_output_level = level;
+	return 0;
+}
+
+double now(void)
+{
+	struct timeval now;     
+	gettimeofday(&now, NULL);       
+	return (double)now.tv_sec + (double)now.tv_usec/1000000.;
+}
+
+size_t dstrftime(char *buf, size_t maxsize, const char *format, double tm)
+{
+	struct timeval tv;
+	double tm_floor;
+	tm_floor = floor(tm);
+	tv.tv_sec = (long) tm_floor;
+	tv.tv_usec = (long) (tm - floor(tm)) * 1000000;
+	return strftime(buf, maxsize, format, localtime((const time_t*) &tv));
+}
+
+#endif

lib/logger.h

@@ -0,0 +1,29 @@
+#include <stdio.h>
+#include <stdarg.h>
+
+#ifndef _LOGGER_H
+#define _LOGGER_H
+
+enum LogLevel { LOG_FATAL, LOG_ERROR, LOG_WARN, LOG_INFO, LOG_DEBUG, LOG_TRACE,
+					NUM_LOGLEVELS };
+
+int log_fatal(const char *loggerName, const char *logMessage, ...) __attribute__((noreturn));
+
+int log_error(const char *loggerName, const char *logMessage, ...);
+
+int log_warn(const char *loggerName, const char *logMessage, ...);
+
+int log_info(const char *loggerName, const char *logMessage, ...);
+
+int log_debug(const char *loggerName, const char *logMessage, ...);
+
+int log_trace(const char *loggerName, const char *logMessage, ...);
+
+int log_init(FILE *stream, enum LogLevel level);
+
+size_t dstrftime(char *, size_t, const char *, double);
+
+double now();
+
+#endif // _LOGGER_H
+

lib/random.c

@@ -0,0 +1,28 @@
+/*
+ * ZMap Copyright 2013 Regents of the University of Michigan 
+ * 
+ * Licensed under the Apache License, Version 2.0 (the "License"); you may not
+ * use this file except in compliance with the License. You may obtain a copy
+ * of the License at http://www.apache.org/licenses/LICENSE-2.0
+ */
+
+#include "random.h"
+#include <stdlib.h>
+#include <stdio.h>
+#include <stdint.h>
+#include <string.h>
+#include <assert.h>
+
+#define RANDSRC "/dev/urandom"
+
+int random_bytes(void *dst, size_t n)
+{
+	FILE *f = fopen(RANDSRC, "rb");
+	assert(f);
+	size_t r = fread(dst, n, 1, f);
+	fclose(f);
+	if (r < 1) {
+		return 0;
+	}
+	return 1;
+}

lib/random.h

@@ -0,0 +1,9 @@
+#include <stdlib.h>
+#include <stdint.h>
+
+#ifndef _RANDOM_H
+#define _RANDOM_H
+
+int random_bytes(void *dst, size_t n);
+
+#endif

lib/redis.c

@@ -0,0 +1,261 @@
+/*
+ * ZMap Redis Helpers Copyright 2013 Regents of the University of Michigan 
+ * 
+ * Licensed under the Apache License, Version 2.0 (the "License"); you may not
+ * use this file except in compliance with the License. You may obtain a copy
+ * of the License at http://www.apache.org/licenses/LICENSE-2.0
+ */
+ 
+#include <string.h>
+#include "redis.h"
+#include <stdlib.h>
+#include <stdio.h>
+#include "assert.h"
+#include "logger.h"
+#include <stdint.h>
+#include <hiredis/hiredis.h>
+
+#define REDIS_UNIX_PATH "/tmp/redis.sock"
+#define REDIS_TIMEOUT 2
+
+#undef  MIN
+#define MIN(X,Y) ((X) < (Y) ? (X) : (Y))
+
+static redisContext *rctx;
+
+static redisContext* redis_connect(void)
+{
+	struct timeval timeout;
+	timeout.tv_sec = REDIS_TIMEOUT;
+	timeout.tv_usec = 0;
+	return (redisContext*) redisConnectUnixWithTimeout(REDIS_UNIX_PATH,
+			timeout);
+}
+
+static int chkerr(redisReply *reply)
+{
+	assert(rctx);
+	if (reply == NULL || reply->type == REDIS_REPLY_ERROR) {
+		log_error("redis", "an error occurred when "
+				"retreiving item from redis: %s",
+				rctx->errstr);
+		if (reply) {
+			freeReplyObject(reply);
+		}
+		return -1;
+	}
+	return 0;
+}
+
+int redis_init(void)
+{
+	rctx = redis_connect();
+	if (!rctx) {
+		return -1;
+	}
+	return 0;
+}
+
+int redis_close(void)
+{
+	redisFree(rctx);
+	return 0;
+}
+
+redisContext* redis_get_context(void)
+{
+	return rctx;
+}
+
+int redis_flush(void)
+{
+	redisReply *reply = (redisReply*) redisCommand(rctx, "FLUSHDB");
+	if (chkerr(reply)) {
+		return -1;
+	}
+	freeReplyObject(reply);
+	return 0;
+}
+
+int redis_existconf(const char *name)
+{
+	assert(rctx);
+	redisReply *reply = (redisReply*) redisCommand(rctx, "EXISTS %s", name);
+	if (chkerr(reply)) {
+		return -1;
+	}
+	int v = reply->integer;
+	freeReplyObject(reply);
+	return v;
+}
+
+int redis_delconf(const char *name)
+{
+	assert(rctx);
+	redisReply *reply = (redisReply*) redisCommand(rctx, "DEL %s", name);
+	if (chkerr(reply)) {
+		return -1;
+	}
+	freeReplyObject(reply);
+	return 0;
+}
+
+int redis_setconf(const char *name, char *value)
+{
+	assert(rctx);
+	redisReply *reply = (redisReply*) redisCommand(rctx, "SET %s %s",
+			name, value);
+	if (chkerr(reply)) {
+		return -1;
+	}
+	freeReplyObject(reply);
+	return 0;
+}
+
+int redis_getconf(const char *name, char *buf, size_t maxlen)
+{
+	assert(rctx);
+	redisReply *reply = (redisReply*) redisCommand(rctx, "GET %s", name);
+	if (chkerr(reply)) {
+		return -1;
+	}
+	strncpy(buf, reply->str, maxlen);
+	freeReplyObject(reply);
+	return 0;
+}
+
+uint32_t redis_getconf_uint32_t(const char *key)
+{
+	assert(rctx);
+	char buf[50];
+	redis_getconf(key, buf, 50);
+	return atoi(buf);
+}
+
+int redis_setconf_uint32_t(const char *key, uint32_t value)
+{
+	assert(rctx);
+	char buf[50];
+	sprintf(buf, "%u", value);
+	return redis_setconf(key, buf);
+}
+
+
+static long redis_get_sizeof(const char *cmd, const char *name)
+{
+	assert(rctx);
+	redisReply *reply;
+	reply = (redisReply*) redisCommand(rctx, "%s %s", cmd, name);
+	assert(reply);
+	assert(reply->type == REDIS_REPLY_INTEGER);
+	long rtr = reply->integer;
+	freeReplyObject(reply);
+	return rtr;
+}
+
+long redis_get_sizeof_list(const char *name)
+{
+	return redis_get_sizeof("LLEN", name);
+}
+
+long redis_get_sizeof_set(const char *name)
+{
+	return redis_get_sizeof("SCARD", name);
+}
+
+int redis_pull(char *redisqueuename, void *buf, 
+		int maxload, size_t obj_size, int *numloaded, const char* cmd)
+{
+	assert(rctx);
+	long elems_in_redis = redis_get_sizeof_list(redisqueuename);
+	long num_to_add = MIN(elems_in_redis, maxload);
+	log_info("redis", "INFO: redis load called on %s. Transfering %li "
+			"of %li elements to in-memory queue.",
+			redisqueuename,
+			num_to_add, elems_in_redis);
+	for(int i=0; i < num_to_add; i++) {
+		redisAppendCommand(rctx, "%s %s", cmd, redisqueuename);
+	}
+	for(int i=0; i < num_to_add; i++) {
+		redisReply *reply;
+		int rc = redisGetReply(rctx, (void**) &reply);
+		if (rc != REDIS_OK) {
+			log_fatal("redis", "response from redis != REDIS_OK");
+			return -1;
+		}
+		if (!reply) {
+			log_fatal("redis", "no reply provided by redis.");
+			return -1;
+		}
+		if (reply->type != REDIS_REPLY_STRING) {
+			log_fatal("redis", 
+					"unxpected reply type from redis.");
+			return -1;
+		}
+		if ((size_t)reply->len != obj_size) {
+			log_fatal("redis", "ERROR: unexpected lengthed "
+					"object provided by redis.\n");
+			return -1;
+		}
+		memcpy((void*)((intptr_t)buf+i*obj_size), reply->str, obj_size);
+		freeReplyObject(reply);
+	}
+	*numloaded = num_to_add;
+	return 0;
+}
+
+int redis_lpull(char *redisqueuename, void *buf, 
+		int maxload, size_t obj_size, int *numloaded)
+{
+	return redis_pull(redisqueuename, buf, 
+			maxload, obj_size, numloaded, "LPOP");
+}
+
+int redis_spull(char *redisqueuename, void *buf, 
+		int maxload, size_t obj_size, int *numloaded)
+{
+	return redis_pull(redisqueuename, buf, 
+			maxload, obj_size, numloaded, "SRAND");
+}
+
+static int redis_push(char *redisqueuename, 
+		void *buf, int num, size_t len, const char *cmd) 
+{
+	assert(rctx);
+	for (int i=0; i < num; i++) {
+		void* load = (void*)((intptr_t)buf + i*len);	
+		int rc = redisAppendCommand(rctx, "%s %s %b", 
+				cmd, redisqueuename, load, len);
+		if (rc != REDIS_OK || rctx->err) {
+			log_fatal("redis", "%s", rctx->errstr);
+			return -1;
+		}
+	}
+	redisReply *reply;
+	for (int i=0; i < num; i++) {
+		if (redisGetReply(rctx, (void**) &reply) != REDIS_OK 
+				|| rctx->err) {
+			log_fatal("redis","%s", rctx->errstr);
+			return -1;
+		}
+		if (reply->type == REDIS_REPLY_ERROR) {
+			log_fatal("redis", "%s", rctx->errstr);
+			return -1;
+		}
+		freeReplyObject(reply);
+	}
+	return 0;
+}
+
+int redis_lpush(char *redisqueuename, 
+		void *buf, int num, size_t len) 
+{
+	return redis_push(redisqueuename, buf, num, len, "RPUSH");
+}
+
+int redis_spush(char *redisqueuename, 
+		void *buf, int num, size_t len) 
+{
+	return redis_push(redisqueuename, buf, num, len, "SADD");
+}
+

lib/redis.h

@@ -0,0 +1,40 @@
+#include <stdint.h>
+#include <unistd.h>
+#include <hiredis/hiredis.h>
+
+#ifndef _REDIS_ZHELPERS_H
+#define _REDIS_ZHELPERS_H
+
+int redis_init(void);
+
+int redis_close(void);
+
+int redis_existconf(const char*);
+
+int redis_flush(void);
+
+int redis_delconf(const char*);
+
+int redis_setconf(const char*, char*);
+
+int redis_getconf(const char*, char*, size_t);
+
+long redis_get_sizeof_list(const char*);
+
+long redis_get_sizeof_set(const char*);
+
+int redis_lpush(char*, void*, int, size_t);
+
+int redis_lpull(char*, void*, int, size_t, int*);
+
+int redis_spull(char*, void*, int, size_t, int*);
+
+int redis_spush(char*, void*, int, size_t);
+
+redisContext* redis_get_context(void);
+
+uint32_t redis_getconf_uint32_t(const char*);
+
+int redis_setconf_uint32_t(const char*, uint32_t);
+
+#endif // _REDIS_ZHELPERS_H

lib/rijndael-alg-fst.h

@@ -0,0 +1,47 @@
+/**
+ * rijndael-alg-fst.h
+ *
+ * @version 3.0 (December 2000)
+ *
+ * Optimised ANSI C code for the Rijndael cipher (now AES)
+ *
+ * @author Vincent Rijmen <vincent.rijmen@esat.kuleuven.ac.be>
+ * @author Antoon Bosselaers <antoon.bosselaers@esat.kuleuven.ac.be>
+ * @author Paulo Barreto <paulo.barreto@terra.com.br>
+ *
+ * This code is hereby placed in the public domain.
+ *
+ * THIS SOFTWARE IS PROVIDED BY THE AUTHORS ''AS IS'' AND ANY EXPRESS
+ * OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE IMPLIED
+ * WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE
+ * ARE DISCLAIMED.  IN NO EVENT SHALL THE AUTHORS OR CONTRIBUTORS BE
+ * LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR
+ * CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF
+ * SUBSTITUTE GOODS OR SERVICES; LOSS OF USE, DATA, OR PROFITS; OR
+ * BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY THEORY OF LIABILITY,
+ * WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT (INCLUDING NEGLIGENCE
+ * OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE OF THIS SOFTWARE,
+ * EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.
+ */
+#ifndef __RIJNDAEL_ALG_FST_H
+#define __RIJNDAEL_ALG_FST_H
+
+#define MAXKC	(256/32)
+#define MAXKB	(256/8)
+#define MAXNR	14
+
+typedef unsigned char	u8;	
+typedef unsigned short	u16;	
+typedef unsigned int	u32;
+
+int rijndaelKeySetupEnc(u32 rk[/*4*(Nr + 1)*/], const u8 cipherKey[], int keyBits);
+int rijndaelKeySetupDec(u32 rk[/*4*(Nr + 1)*/], const u8 cipherKey[], int keyBits);
+void rijndaelEncrypt(const u32 rk[/*4*(Nr + 1)*/], int Nr, const u8 pt[16], u8 ct[16]);
+void rijndaelDecrypt(const u32 rk[/*4*(Nr + 1)*/], int Nr, const u8 ct[16], u8 pt[16]);
+
+#ifdef INTERMEDIATE_VALUE_KAT
+void rijndaelEncryptRound(const u32 rk[/*4*(Nr + 1)*/], int Nr, u8 block[16], int rounds);
+void rijndaelDecryptRound(const u32 rk[/*4*(Nr + 1)*/], int Nr, u8 block[16], int rounds);
+#endif /* INTERMEDIATE_VALUE_KAT */
+
+#endif /* __RIJNDAEL_ALG_FST_H */

src/Makefile

@@ -0,0 +1,66 @@
+CC=gcc
+
+CFLAGS=-Wall -pedantic -Wextra -std=gnu99 -I../lib -I./ -Ioutput_modules -O2 -g
+LDFLAGS=-g -pthread
+LDLIBS= -lpcap -lgmp -lm
+TARGETS=zmap
+VPATH=../lib:output_modules:probe_modules
+PREFIX=/usr/local
+
+INSTALL=install
+INSTALLDATA=install -m 644
+mandir=/usr/share/man/man1/
+bindir=$(PREFIX)/sbin
+
+# Hardening and warnings for building with gcc
+#M aybe add -Wstrict-prototypes -Wmissing-prototypes -Wmissing-declarations
+#-Wold-style-definition -Wswitch-enum 
+GCCWARNINGS = -Wall -fno-strict-aliasing -W -Wfloat-equal -Wundef      \
+-Wpointer-arith														   \
+-Wwrite-strings -Wredundant-decls -Wchar-subscripts -Wcomment          \
+-Wformat=2 -Wwrite-strings -Wredundant-decls -Wno-unused-but-set-variable    \
+-Wnested-externs -Wbad-function-cast -Winit-self         \
+-Wmissing-field-initializers										   \
+-Waddress -Wmissing-noreturn -Wnormalized=id    \
+-Woverride-init -Wstrict-overflow=1 -Wextra -Warray-bounds             \
+-Wstack-protector -Wformat -Wformat-security -Wpointer-sign -Wno-format-nonliteral -Wno-format-y2k
+GCCHARDENING=-D_FORTIFY_SOURCE=2 -fstack-protector-all -fwrapv -fPIC --param ssp-buffer-size=1
+LDHARDENING=-z relro -z now
+
+EXTRACFLAGS=-g -O2 $(EXTRA_CFLAGS) $(GCCHARDENING) $(GCCWARNINGS) -Werror
+EXTRALDFLAGS= $(LDHARDENING)
+
+CFLAGS+=-Wall $(INCLUDE) $(EXTRACFLAGS)
+LDFLAGS+=$(EXTRALDFLAGS)
+
+modules=module_tcp_synscan.o module_icmp_echo.o module_udp.o #ADD YOUR MODULE HERE
+
+objects=constraint.o blacklist.o cyclic.o logger.o send.o recv.o state.o monitor.o zopt.o zmap.o random.o output_modules.o module_simple_file.o module_extended_file.o packet.o probe_modules.o ${modules} validate.o rijndael-alg-fst.o get_gateway.o aesrand.o
+
+ifeq ($(REDIS), true)
+	LDLIBS+=-lhiredis
+	objects+=module_redis.o redis.o module_ssldb.o
+	CFLAGS+=-DREDIS
+endif
+
+all: $(TARGETS)
+
+$(TARGETS):
+	$(CC) $(CFLAGS) $(DFLAGS) $(LDFLAGS) $^ -o $@ $(LDLIBS)
+
+zmap: $(objects) 
+
+zopt.c zopt.h: zopt.ggo
+	gengetopt -C --no-help --no-version -i $^ -F $*
+
+install: zmap
+	$(INSTALL) zmap $(bindir)/zmap
+	test -d /etc/zmap || (mkdir /etc/zmap && $(INSTALLDATA) ../conf/* /etc/zmap/)
+	$(INSTALLDATA) ./zmap.1 $(mandir)
+	echo "\n\n\n\n**************\nSuccess! ZMap is installed. Try running (as root):\nzmap -p 80 -N 10 -B 1M -o -\n**************"
+
+clean:
+	-rm -f $(objects) $(TARGETS)
+
+.PHONY: install clean
+

src/aesrand.c

@@ -0,0 +1,57 @@
+/*
+ * ZMap Copyright 2013 Regents of the University of Michigan 
+ * 
+ * Licensed under the Apache License, Version 2.0 (the "License"); you may not
+ * use this file except in compliance with the License. You may obtain a copy
+ * of the License at http://www.apache.org/licenses/LICENSE-2.0
+ */
+
+#include <stdlib.h>
+#include <stdint.h>
+#include <assert.h>
+#include <string.h>
+
+#include "../lib/rijndael-alg-fst.h"
+#include "../lib/random.h"
+#include "../lib/logger.h"
+
+#define AES_ROUNDS 10
+#define AES_BLOCK_WORDS  4
+#define AES_KEY_BYTES 16
+#define OUTPUT_BYTES 16
+
+static uint32_t aes_input[AES_BLOCK_WORDS];
+static uint32_t aes_sched[(AES_ROUNDS+1)*4];
+static uint8_t aes_output[OUTPUT_BYTES];
+static int init = 0;
+
+void aesrand_init(uint32_t seed)
+{
+	memset(&aes_input, 0, sizeof(aes_input));
+	uint8_t key[AES_KEY_BYTES];
+	if (seed) {
+		memset(key, 0, AES_KEY_BYTES*sizeof(uint8_t));
+		memcpy(key, &seed, sizeof(uint32_t));
+	} else {
+		if (!random_bytes(key, AES_KEY_BYTES)) {
+    		log_fatal("aesrand", "couldn't get random bytes");
+  		}
+	}
+	if (rijndaelKeySetupEnc(aes_sched, key, AES_KEY_BYTES*8) != AES_ROUNDS) {
+		log_fatal("aesrand", "could not initialize AES key");
+	}
+	memset(aes_output, 0, OUTPUT_BYTES*sizeof(uint8_t));
+	init = 1;
+}
+
+uint64_t aesrand_getword(void)
+{
+	assert(init);
+	memcpy(aes_input, aes_output, sizeof(aes_input));
+	rijndaelEncrypt(aes_sched, AES_ROUNDS,
+		(uint8_t *)aes_input, aes_output);
+	uint64_t retval;
+	memcpy(&retval, aes_output, sizeof(retval)); 
+	return retval;
+}
+

src/aesrand.h

@@ -0,0 +1,18 @@
+/*
+ * ZMap Copyright 2013 Regents of the University of Michigan 
+ * 
+ * Licensed under the Apache License, Version 2.0 (the "License"); you may not
+ * use this file except in compliance with the License. You may obtain a copy
+ * of the License at http://www.apache.org/licenses/LICENSE-2.0
+ */
+
+#include <stdint.h>
+
+#ifndef _AESRAND_H
+#define _AESRAND_H
+
+void aesrand_init(uint32_t seed);
+
+uint64_t aesrand_getword(void);
+
+#endif

src/cyclic.c

@@ -0,0 +1,184 @@
+/*
+ * ZMap Copyright 2013 Regents of the University of Michigan 
+ * 
+ * Licensed under the Apache License, Version 2.0 (the "License"); you may not
+ * use this file except in compliance with the License. You may obtain a copy
+ * of the License at http://www.apache.org/licenses/LICENSE-2.0
+ */
+
+/*
+ * cyclic provides an inexpensive approach to iterating over the IPv4 address
+ * space in a random(-ish) manner such that we connect to every host once in
+ * a scan execution without having to keep track of the IPs that have been
+ * scanned or need to be scanned and such that each scan has a different 
+ * ordering. We accomplish this by utilizing a cyclic multiplicative group 
+ * of integers modulo a prime and generating a new primitive root (generator)
+ * for each scan.
+ *
+ * We know that 3 is a generator of (Z mod 2^32 + 15 - {0}, *) 
+ * and that we have coverage over the entire address space because 2**32 + 15
+ * is prime and ||(Z mod PRIME - {0}, *)|| == PRIME - 1. Therefore, we
+ * just need to find a new generator (primitive root) of the cyclic group for
+ * each scan that we perform.
+ *
+ * Because generators map to generators over an isomorphism, we can efficiently
+ * find random primitive roots of our mult. group by finding random generators
+ * of the group (Zp-1, +) which is isomorphic to (Zp*, *). Specifically the
+ * generators of (Zp-1, +) are { s | (s, p-1) == 1 } which implies that
+ * the generators of (Zp*, *) are { d^s | (s, p-1) == 1 }. where d is a known
+ * generator of the multiplicative group. We efficiently find
+ * generators of the additive group by precalculating the psub1_f of
+ * p - 1 and randomly checking random numbers against the psub1_f until
+ * we find one that is coprime and map it into Zp*. Because
+ * totient(totient(p)) ~= 10^9, this should take relatively few
+ * iterations to find a new generator. 
+ */
+
+#include "cyclic.h"
+
+#include <stdlib.h>
+#include <stdio.h>
+#include <stdint.h>
+#include <time.h>
+#include <assert.h>
+#include <string.h>
+#include <math.h>
+
+#include <arpa/inet.h>
+#include <sys/socket.h>
+#include <netinet/in.h>
+
+#include <gmp.h>
+
+#include "../lib/logger.h"
+#include "../lib/blacklist.h"
+
+#include "state.h"
+#include "aesrand.h"
+
+#define LSRC "cyclic"
+#define PRIME 4294967311 // 2^32 + 15
+#define KNOWN_PRIMROOT 3
+
+// distinct prime factors of 2^32 + 15
+static const uint64_t psub1_f[] = { 2, 3, 5, 131, 364289 }; 
+
+// selected primitive root that we'll use as the generator
+static uint64_t primroot = 0;
+static uint64_t current = 0;
+
+#define COPRIME 1
+#define NOT_COPRIME 0 
+
+// check whether two integers are coprime
+static int check_coprime(uint64_t check)
+{
+	for (unsigned i=0; i < sizeof(psub1_f)/sizeof(psub1_f[0]); i++) {
+		if (psub1_f[i] > check && !(psub1_f[i] % check)) {
+			return NOT_COPRIME;
+		} else if (psub1_f[i] < check && !(check % psub1_f[i])) {
+			return NOT_COPRIME;
+		} else if (psub1_f[i] == check) {
+			return NOT_COPRIME;
+		}
+	}
+	return COPRIME;
+}
+
+// find gen of cyclic group Z modulo PRIME
+static uint64_t find_primroot(void)
+{
+	// what luck, rand() returns a uint32_t!
+	uint32_t candidate = (uint32_t) aesrand_getword() & 0xFFFF;
+	while(check_coprime(candidate) != COPRIME) {
+		++candidate;
+	}
+	// pre-modded result is gigantic so use GMP
+	mpz_t base, power, prime, primroot;
+	mpz_init_set_d(base, (double) KNOWN_PRIMROOT);
+	mpz_init_set_d(power, (double) candidate);
+	mpz_init_set_d(prime, (double) PRIME);
+	mpz_init(primroot);
+	mpz_powm(primroot, base, power, prime);
+	uint64_t retv = (uint64_t) mpz_get_ui(primroot);
+	mpz_clear(base);
+	mpz_clear(power);
+	mpz_clear(prime);
+	mpz_clear(primroot);
+	return retv;
+}
+
+int cyclic_init(uint32_t primroot_, uint32_t current_)
+{
+	assert(!(!primroot_ && current_));
+
+	if (zconf.use_seed) {
+		aesrand_init(zconf.seed+1);
+	} else {
+		aesrand_init(0);
+	}
+	if (!primroot_) {
+		do {
+			primroot = find_primroot();
+		} while (primroot >= (1LL << 32));
+		log_debug(LSRC, "primitive root: %lld", primroot);
+		current = (uint32_t) aesrand_getword() & 0xFFFF;
+		log_debug(LSRC, "starting point: %lld", current);
+	} else {
+		primroot = primroot_;
+		log_debug(LSRC, "primitive root %lld specified by caller",
+				primroot);
+		if (!current_) {
+			current = (uint32_t) aesrand_getword() & 0xFFFF;
+			log_debug(LSRC, "no cyclic starting point, "
+					 "selected random startpoint: %lld",
+					 current);
+		} else {
+			current = current_;
+		    log_debug(LSRC, "starting point %lld specified by caller",
+				    current);
+		}
+	}
+	zconf.generator = primroot;
+	if (blacklist_init_from_files(zconf.whitelist_filename,
+							zconf.blacklist_filename)) {
+	  return -1;
+	}
+
+	// make sure current is an allowed ip
+	cyclic_get_next_ip();
+
+	return 0;
+}
+
+uint32_t cyclic_get_curr_ip(void)
+{
+	return (uint32_t) current;
+}
+
+uint32_t cyclic_get_primroot(void)
+{
+	return (uint32_t) primroot;
+}
+
+static inline uint32_t cyclic_get_next_elem(void)
+{
+	do {
+		current *= primroot;
+		current %= PRIME;
+	} while (current >= (1LL << 32));
+	return (uint32_t) current;
+}
+
+uint32_t cyclic_get_next_ip(void)
+{
+	while (1) {
+		uint32_t candidate = cyclic_get_next_elem();
+		if (!blacklist_is_allowed(candidate)) {
+			zsend.blacklisted++;
+		} else {
+			return candidate;
+		}
+	}
+}
+

src/cyclic.h

@@ -0,0 +1,25 @@
+/*
+ * ZMap Copyright 2013 Regents of the University of Michigan 
+ * 
+ * Licensed under the Apache License, Version 2.0 (the "License"); you may not
+ * use this file except in compliance with the License. You may obtain a copy
+ * of the License at http://www.apache.org/licenses/LICENSE-2.0
+ */
+
+#include <stdint.h>
+
+#ifndef _CYCLIC_H
+#define _CYCLIC_H
+
+int cyclic_init(uint32_t, uint32_t);
+
+// get next IP address to scan
+uint32_t cyclic_get_next_ip(void);
+
+// what IP address was returned last
+uint32_t cyclic_get_curr_ip(void);
+
+// what primitive root was generated for this current scan
+uint32_t cyclic_get_primroot(void);
+
+#endif

src/get_gateway.c

@@ -0,0 +1,237 @@
+/*
+ * ZMap Copyright 2013 Regents of the University of Michigan 
+ * 
+ * Licensed under the Apache License, Version 2.0 (the "License"); you may not
+ * use this file except in compliance with the License. You may obtain a copy
+ * of the License at http://www.apache.org/licenses/LICENSE-2.0
+ */
+
+#include <stdlib.h>
+#include <stdio.h>
+#include <string.h>
+#include <unistd.h>
+#include <errno.h>
+#include <assert.h>
+
+#include <netinet/in.h>
+#include <net/if.h>
+#include <sys/socket.h>
+#include <sys/ioctl.h>
+#include <linux/netlink.h>
+#include <linux/rtnetlink.h>
+#include <sys/types.h>
+#include <sys/socket.h>
+#include <arpa/inet.h>
+
+#include "../lib/logger.h"
+
+int read_nl_sock(int sock, char *buf, int buf_len)
+{
+	int msg_len = 0;
+	char *pbuf = buf;
+	do {
+		int len = recv(sock, pbuf, buf_len - msg_len, 0);
+		if (len <= 0) {
+			log_debug("get-gw", "recv failed: %s", strerror(errno));
+			return -1;
+		}
+		struct nlmsghdr *nlhdr = (struct nlmsghdr *)pbuf;
+		if (NLMSG_OK(nlhdr, ((unsigned int)len)) == 0 || 
+						nlhdr->nlmsg_type == NLMSG_ERROR) {
+			log_debug("get-gw", "recv failed: %s", strerror(errno));
+			return -1;
+		}
+		if (nlhdr->nlmsg_type == NLMSG_DONE) {
+			break;
+		} else {
+			msg_len += len;
+			pbuf += len;
+		} 
+		if ((nlhdr->nlmsg_flags & NLM_F_MULTI) == 0) {
+			break;
+		}
+	} while (1);
+	return msg_len;
+}
+
+int send_nl_req(uint16_t msg_type, uint32_t seq,
+				void *payload, uint32_t payload_len)
+{
+	int sock = socket(PF_NETLINK, SOCK_DGRAM, NETLINK_ROUTE);
+	if (sock < 0) {
+		log_error("get-gw", "unable to get socket: %s", strerror(errno));
+		return -1;
+	}
+	if (NLMSG_SPACE(payload_len) < payload_len) {
+		// Integer overflow
+		return -1;
+	}
+	struct nlmsghdr *nlmsg;
+	nlmsg = malloc(NLMSG_SPACE(payload_len));
+	if (!nlmsg) {
+		return -1;
+	}
+
+	memset(nlmsg, 0, sizeof(nlmsg));
+	memcpy(NLMSG_DATA(nlmsg), payload, payload_len);
+	nlmsg->nlmsg_type = msg_type;
+	nlmsg->nlmsg_len = NLMSG_LENGTH(payload_len);
+	nlmsg->nlmsg_flags = NLM_F_DUMP | NLM_F_REQUEST;
+	nlmsg->nlmsg_seq = seq;
+	nlmsg->nlmsg_pid = getpid();
+
+	if (send(sock, nlmsg, nlmsg->nlmsg_len, 0) < 0) {
+		log_error("get-gw", "failure sending: %s", strerror(errno));
+		return -1;
+	}
+	free(nlmsg);
+	return sock;
+}
+
+int get_hw_addr(struct in_addr *gw_ip, char *iface, unsigned char *hw_mac)
+{
+	char buf[8192];
+	struct ndmsg req;
+	struct nlmsghdr *nlhdr;
+
+	if (!gw_ip || !hw_mac) {
+		return -1;
+	}
+	// Send RTM_GETNEIGH request
+	req.ndm_family = AF_INET;
+	req.ndm_ifindex = if_nametoindex(iface);
+	req.ndm_state = NUD_REACHABLE;
+	req.ndm_type = NDA_LLADDR;
+
+	int sock = send_nl_req(RTM_GETNEIGH, 1, &req, sizeof(req));
+
+	// Read responses
+	unsigned nl_len = read_nl_sock(sock, buf, sizeof(buf));
+	if (nl_len <= 0) {
+		return -1;
+	}
+	// Parse responses
+	nlhdr = (struct nlmsghdr *)buf;
+	while (NLMSG_OK(nlhdr, nl_len)) {
+		struct rtattr *rt_attr;
+		struct rtmsg *rt_msg;
+		int rt_len;
+		unsigned char mac[6];
+		struct in_addr dst_ip;
+		int correct_ip = 0;
+
+		rt_msg = (struct rtmsg *) NLMSG_DATA(nlhdr);
+		
+		if ((rt_msg->rtm_family != AF_INET)) {
+			return -1;
+		}
+	
+		rt_attr = (struct rtattr *) RTM_RTA(rt_msg);
+		rt_len = RTM_PAYLOAD(nlhdr);
+		while (RTA_OK(rt_attr, rt_len)) {
+			switch (rt_attr->rta_type) {
+			case NDA_LLADDR: 
+				assert(RTA_PAYLOAD(rt_attr) == IFHWADDRLEN);
+				memcpy(mac, RTA_DATA(rt_attr), IFHWADDRLEN);
+				break;
+			case NDA_DST:
+				assert(RTA_PAYLOAD(rt_attr) == sizeof(dst_ip));
+				memcpy(&dst_ip, RTA_DATA(rt_attr), sizeof(dst_ip));
+				if (memcmp(&dst_ip, gw_ip, sizeof(dst_ip)) == 0) {
+					correct_ip = 1;
+				}
+				break;
+			}
+			rt_attr = RTA_NEXT(rt_attr, rt_len);
+		}
+		if (correct_ip) {
+			memcpy(hw_mac, mac, IFHWADDRLEN);
+			return 0;
+		}
+		nlhdr = NLMSG_NEXT(nlhdr, nl_len);	
+	}
+	return -1;				
+}
+
+// gw and iface[IF_NAMESIZE] MUST be allocated
+int get_default_gw(struct in_addr *gw, char *iface)
+{
+	struct rtmsg req;
+	unsigned int nl_len;
+	char buf[8192];
+	struct nlmsghdr *nlhdr;
+
+	if (!gw || !iface) {
+		return -1;
+	}
+
+	// Send RTM_GETROUTE request
+	memset(&req, 0, sizeof(req));
+	int sock = send_nl_req(RTM_GETROUTE, 0, &req, sizeof(req));
+
+	// Read responses
+	nl_len = read_nl_sock(sock, buf, sizeof(buf));
+	if (nl_len <= 0) {
+		return -1;
+	}
+
+	// Parse responses
+	nlhdr = (struct nlmsghdr *)buf;
+	while (NLMSG_OK(nlhdr, nl_len)) {
+		struct rtattr *rt_attr;
+		struct rtmsg *rt_msg;
+		int rt_len;
+		int has_gw = 0;
+
+		rt_msg = (struct rtmsg *) NLMSG_DATA(nlhdr);
+		
+		if ((rt_msg->rtm_family != AF_INET) || (rt_msg->rtm_table != RT_TABLE_MAIN)) {
+			return -1;
+		}
+	
+		rt_attr = (struct rtattr *) RTM_RTA(rt_msg);
+		rt_len = RTM_PAYLOAD(nlhdr);
+		while (RTA_OK(rt_attr, rt_len)) {
+			switch (rt_attr->rta_type) {
+			case RTA_OIF:
+				if_indextoname(*(int *) RTA_DATA(rt_attr), iface); 
+				break;
+			case RTA_GATEWAY:
+				gw->s_addr = *(unsigned int *) RTA_DATA(rt_attr); 
+				has_gw = 1;
+				break;
+			}
+			rt_attr = RTA_NEXT(rt_attr, rt_len);
+		}
+	
+		if (has_gw) {
+			return 0;
+		}
+		nlhdr = NLMSG_NEXT(nlhdr, nl_len);	
+	}
+	return -1;
+}
+
+// Returns the first IP address for a given iface
+int get_iface_ip(char *iface, struct in_addr *ip)
+{
+	int sock;
+	struct ifreq ifr;
+	sock = socket(AF_INET, SOCK_DGRAM, 0);
+	if (sock < 0) {
+		log_error("get-gw", "failure opening socket: %s", strerror(errno));
+		return -1;
+	}
+	ifr.ifr_addr.sa_family = AF_INET;
+	strncpy(ifr.ifr_name, iface, IFNAMSIZ-1);
+
+	if (ioctl(sock, SIOCGIFADDR, &ifr) < 0) {
+		log_error("get-gw", "ioctl failure: %s", strerror(errno));
+		close(sock);
+		return -1;
+	}
+	close(sock);
+	memcpy(ip, &((struct sockaddr_in*)&ifr.ifr_addr)->sin_addr, sizeof(*ip));
+	return 0;
+}
+

src/get_gateway.h

@@ -0,0 +1,18 @@
+/*
+ * ZMap Copyright 2013 Regents of the University of Michigan 
+ * 
+ * Licensed under the Apache License, Version 2.0 (the "License"); you may not
+ * use this file except in compliance with the License. You may obtain a copy
+ * of the License at http://www.apache.org/licenses/LICENSE-2.0
+ */
+
+#ifndef _GET_GATEWAY_H
+#define _GET_GATEWAY_H
+
+#include <netinet/in.h>
+
+int get_hw_addr(struct in_addr *gw_ip, char *iface, unsigned char *hw_mac);
+int get_default_gw(struct in_addr *gw, char *iface);
+int get_iface_ip(char *iface, struct in_addr *ip);
+
+#endif

src/monitor.c

@@ -0,0 +1,231 @@
+/*
+ * ZMap Copyright 2013 Regents of the University of Michigan 
+ * 
+ * Licensed under the Apache License, Version 2.0 (the "License"); you may not
+ * use this file except in compliance with the License. You may obtain a copy
+ * of the License at http://www.apache.org/licenses/LICENSE-2.0
+ */
+
+// module responsible for printing on-screen updates during the scan process
+
+#include <stdlib.h>
+#include <stdio.h>
+#include <unistd.h>
+#include <math.h>
+
+#include "recv.h"
+#include "monitor.h"
+#include "state.h"
+
+#include "../lib/logger.h"
+
+#define UPDATE_INTERVAL 1 //seconds
+
+static double last_now = 0.0;
+static uint32_t last_sent = 0;
+static uint32_t last_rcvd = 0;
+static uint32_t last_drop = 0;
+static uint32_t last_failures = 0;
+
+static double min_d(double array[], int n)
+{
+	double value=INFINITY;
+	for (int i=0; i<n; i++) {
+		if (array[i] < value) {
+			value = array[i];
+		}
+	}
+	return value;
+}
+
+// pretty print elapsed (or estimated) number of seconds
+static void time_string(uint32_t time, int est, char *buf, size_t len)
+{
+  	int y = time / 31556736;
+  	int d = (time % 31556736) / 86400;
+	int h = (time % 86400) / 3600;
+	int m = (time % 3600) / 60;
+	int s = time % 60;
+
+	if (est) {
+		if (y > 0) {
+			snprintf(buf, len, "%d years", y);
+		} else if (d > 9) {
+			snprintf(buf, len, "%dd", d);
+		} else if (d > 0) {
+			snprintf(buf, len, "%dd%02dh", d, h);
+		} else if (h > 9) {
+			snprintf(buf, len, "%dh", h);
+		} else if (h > 0) {
+			snprintf(buf, len, "%dh%02dm", h, m);
+		} else if (m > 9) {
+			snprintf(buf, len, "%dm", m);
+		} else if (m > 0) {
+			snprintf(buf, len, "%dm%02ds", m, s);
+		} else {
+			snprintf(buf, len, "%ds", s);
+		}
+	} else {
+		if (d > 0) {
+			snprintf(buf, len, "%dd%d:%02d:%02d", d, h, m, s);
+		} else if (h > 0) {
+			snprintf(buf, len, "%d:%02d:%02d", h, m, s);
+		} else {
+			snprintf(buf, len, "%d:%02d", m, s);
+		}
+	}
+}
+
+// pretty print quantities
+static void number_string(uint32_t n, char *buf, size_t len)
+{
+	int figs = 0;
+	if (n < 1000) {
+		snprintf(buf, len, "%u ", n);
+	} else if (n < 1000000) {
+		if (n < 10000) {
+		figs = 2;
+		} else if (n < 100000) {
+		figs = 1;
+		}
+		snprintf(buf, len, "%0.*f K", figs, (float)n/1000.);
+	} else {
+		if (figs < 10000000) {
+		figs = 2;
+		} else if (figs < 100000000) {
+			figs = 1;
+		}
+		snprintf(buf, len, "%0.*f M", figs, (float)n/1000000.);
+	}
+}
+
+// estimate time remaining time based on config and state
+double compute_remaining_time(double age)
+{
+	if (!zsend.complete) {
+		double remaining[] = {INFINITY, INFINITY, INFINITY};
+		if (zsend.targets) {
+			double done = (double)zsend.sent/zsend.targets;
+			remaining[0] = (1. - done)*(age/done) + zconf.cooldown_secs;
+		}
+		if (zconf.max_runtime) {
+			remaining[1] = (zconf.max_runtime - age)+zconf.cooldown_secs;
+		}
+		if (zconf.max_results) {
+			double done = (double)zrecv.success_unique/zconf.max_results;
+			remaining[2] = (1. - done)*(age/done);
+		}
+		return min_d(remaining, sizeof(remaining)/sizeof(double));
+	} else {
+		return zconf.cooldown_secs - (now() - zsend.finish);
+	}
+}
+
+static void monitor_update(void)
+{
+	if (last_now > 0.0) {
+		double age = now() - zsend.start;
+		double delta = now() - last_now;
+		double remaining_secs = compute_remaining_time(age);
+		double percent_complete = 100.*age/(age + remaining_secs);
+
+		// ask pcap for fresh values
+		recv_update_pcap_stats();
+
+		// format times for display
+		char time_left[20];
+		if (age < 5) {
+			time_left[0] = '\0';
+		} else {
+			char buf[20];
+			time_string((int)remaining_secs, 1, buf, sizeof(buf));
+			snprintf(time_left, sizeof(time_left), " (%s left)", buf);
+		}
+		char time_past[20];
+		time_string((int)age, 0, time_past, sizeof(time_past));
+
+		char send_rate[20], send_avg[20],
+			 recv_rate[20], recv_avg[20],
+			 pcap_drop[20], pcap_drop_avg[20];
+		// recv stats
+		number_string((zrecv.success_unique - last_rcvd)/delta,
+						recv_rate, sizeof(recv_rate));
+		number_string((zrecv.success_unique/age), recv_avg, sizeof(recv_avg));
+		// dropped stats
+		number_string((zrecv.pcap_drop + zrecv.pcap_ifdrop - last_drop)/delta,
+						pcap_drop, sizeof(pcap_drop));
+		number_string(((zrecv.pcap_drop + zrecv.pcap_ifdrop)/age),
+						pcap_drop_avg, sizeof(pcap_drop_avg));
+
+		// Warn if we drop > 5% of our average receive rate
+		uint32_t drop_rate = (uint32_t)((zrecv.pcap_drop + zrecv.pcap_ifdrop - last_drop) / delta);
+		if (drop_rate > (uint32_t)((zrecv.success_unique - last_rcvd) / delta) / 20) {
+			log_warn("monitor", "Dropped %d packets in the last second, (%d total dropped (pcap: %d + iface: %d))",
+					 drop_rate, zrecv.pcap_drop + zrecv.pcap_ifdrop, zrecv.pcap_drop, zrecv.pcap_ifdrop);
+		}	
+
+		// Warn if we fail to send > 1% of our average send rate
+		uint32_t fail_rate = (uint32_t)((zsend.sendto_failures - last_failures) / delta); // failures/sec
+		if (fail_rate > ((zsend.sent / age) / 100)) {
+			log_warn("monitor", "Failed to send %d packets/sec (%d total failures)",
+					 fail_rate, zsend.sendto_failures);
+		}
+
+		if (!zsend.complete) {
+			// main display (during sending)
+			number_string((zsend.sent - last_sent)/delta,
+							send_rate, sizeof(send_rate));
+			number_string((zsend.sent/age), send_avg, sizeof(send_avg));
+			fprintf(stderr,
+					"%5s %0.0f%%%s; send: %u %sp/s (%sp/s avg); "
+					"recv: %u %sp/s (%sp/s avg); "
+					"drops: %sp/s (%sp/s avg); "
+					"hits: %0.2f%%\n", 
+					time_past,
+					percent_complete,
+					time_left,
+					zsend.sent,
+					send_rate,
+					send_avg,
+					zrecv.success_unique,
+					recv_rate,
+					recv_avg,
+					pcap_drop,
+					pcap_drop_avg,
+					zrecv.success_unique*100./zsend.sent);
+		} else {
+		  	// alternate display (during cooldown)
+			number_string((zsend.sent/(zsend.finish - zsend.start)), send_avg, sizeof(send_avg));
+			fprintf(stderr, 
+					"%5s %0.0f%%%s; send: %u done (%sp/s avg); "
+					"recv: %u %sp/s (%sp/s avg); "
+					"drops: %sp/s (%sp/s avg); "
+					"hits: %0.2f%%\n", 
+					time_past,
+					percent_complete,
+					time_left,
+					zsend.sent,
+					send_avg,
+					zrecv.success_unique,
+					recv_rate,
+					recv_avg,
+					pcap_drop,
+					pcap_drop_avg,
+					zrecv.success_unique*100./zsend.sent);
+		}
+	}
+	last_now  = now();
+	last_sent = zsend.sent;
+	last_rcvd = zrecv.success_unique;
+	last_drop = zrecv.pcap_drop + zrecv.pcap_ifdrop;
+	last_failures = zsend.sendto_failures;
+}
+
+void monitor_run(void)
+{
+	while (!(zsend.complete && zrecv.complete))  {
+		monitor_update();
+		sleep(UPDATE_INTERVAL);
+	}
+}
+

src/monitor.h

@@ -0,0 +1,14 @@
+/*
+ * ZMap Copyright 2013 Regents of the University of Michigan 
+ * 
+ * Licensed under the Apache License, Version 2.0 (the "License"); you may not
+ * use this file except in compliance with the License. You may obtain a copy
+ * of the License at http://www.apache.org/licenses/LICENSE-2.0
+ */
+
+#ifndef _MONITOR_H
+#define _MONITOR_H
+
+void monitor_run();
+
+#endif

src/output_modules/module_extended_file.c

@@ -0,0 +1,109 @@
+/*
+ * ZMap Copyright 2013 Regents of the University of Michigan 
+ * 
+ * Licensed under the Apache License, Version 2.0 (the "License"); you may not
+ * use this file except in compliance with the License. You may obtain a copy
+ * of the License at http://www.apache.org/licenses/LICENSE-2.0
+ */
+
+#include <stdio.h>
+#include <stdlib.h>
+#include <string.h>
+#include <time.h>
+#include <assert.h>
+
+#include <sys/time.h>
+#include <sys/socket.h>
+#include <netinet/in.h>
+#include <arpa/inet.h>
+
+#include "../../lib/logger.h"
+
+#include "output_modules.h"
+
+static FILE *file = NULL;
+#define UNUSED __attribute__((unused))
+
+int extendedfile_init(struct state_conf *conf)
+{
+	assert(conf);
+	if (conf->output_filename) {
+		if (!strcmp(conf->output_filename, "-")) {
+			file = stdout;
+		} else {
+			if (!(file = fopen(conf->output_filename, "w"))) {
+				perror("Couldn't open output file");
+				exit(EXIT_FAILURE);
+			}
+		}
+		fprintf(file, "response, saddr, daddr, sport, "
+						"dport, seq, ack, in_cooldown, is_repeat, timestamp\n");
+	}
+	return EXIT_SUCCESS;
+}
+
+static void fprint_tv(FILE *f, struct timeval *tv) 
+{
+	char time_string[40]; 
+	struct tm *ptm = localtime(&tv->tv_sec); 
+	strftime(time_string, sizeof (time_string),
+					"%Y-%m-%d %H:%M:%S", ptm);
+	long milliseconds = tv->tv_usec / 1000; 
+	fprintf(f, "%s.%03ld\n", time_string, milliseconds); 
+} 
+
+
+int extendedfile_ip(ipaddr_n_t saddr, ipaddr_n_t daddr,
+		const char *response_type, int is_repeat,
+		int in_cooldown, const u_char *packet, size_t buflen)
+{
+	struct iphdr *ip_hdr = (struct iphdr *)&packet[sizeof(struct ethhdr)];
+	if (buflen < (sizeof(struct ethhdr) + ip_hdr->ihl*4 + sizeof(struct tcphdr)))
+		return EXIT_FAILURE;
+	struct tcphdr *tcp = (struct tcphdr *)((char *)ip_hdr + ip_hdr->ihl * 4);
+
+	if (file) {
+		struct in_addr addr;
+		addr.s_addr = saddr;
+		// inet_ntoa returns a <<const>> char * 
+		fprintf(file, "%s, %s, ", 
+				response_type, 
+				inet_ntoa(addr));
+		addr.s_addr = daddr;
+		fprintf(file, "%s, %u, %u, %u, %u, %i, %i,", 
+				inet_ntoa(addr),
+				ntohs(tcp->source),
+				ntohs(tcp->dest),
+				ntohl(tcp->seq),
+				ntohl(tcp->ack_seq),
+				in_cooldown,
+				is_repeat);
+		struct timeval t;
+		gettimeofday(&t, NULL);
+		fprint_tv(file, &t);
+		fflush(file);
+	}
+	return EXIT_SUCCESS;
+}
+
+int extendedfile_close(UNUSED struct state_conf* c, 
+		UNUSED struct state_send* s, UNUSED struct state_recv* r)
+{
+	if (file) {
+		fflush(file);
+		fclose(file);
+	}
+	return EXIT_SUCCESS;
+}
+
+output_module_t module_extended_file = {
+	.name = "extended_file",
+	.init = &extendedfile_init,
+	.start = NULL,
+	.update = NULL,
+	.update_interval = 0,
+	.close = &extendedfile_close,
+	.success_ip = &extendedfile_ip,
+	.other_ip = &extendedfile_ip
+};
+

src/output_modules/module_extended_file.h

@@ -0,0 +1,20 @@
+/*
+ * ZMap Copyright 2013 Regents of the University of Michigan 
+ * 
+ * Licensed under the Apache License, Version 2.0 (the "License"); you may not
+ * use this file except in compliance with the License. You may obtain a copy
+ * of the License at http://www.apache.org/licenses/LICENSE-2.0
+ */
+
+#include <output_modules.h>
+
+int extendedfile_init(struct state_conf *conf);
+
+int extendedfile_ip(ipaddr_n_t saddr, ipaddr_n_t daddr,
+        port_n_t sport, port_n_t dport, struct timeval* t,
+        const char *response_type, int is_repeat,
+        int in_cooldown, const u_char *packet);
+
+int extendedfile_close(struct state_conf* c, struct state_send* s,
+		struct state_recv* r);
+

src/output_modules/module_redis.c

@@ -0,0 +1,96 @@
+/*
+ * ZMap Copyright 2013 Regents of the University of Michigan 
+ * 
+ * Licensed under the Apache License, Version 2.0 (the "License"); you may not
+ * use this file except in compliance with the License. You may obtain a copy
+ * of the License at http://www.apache.org/licenses/LICENSE-2.0
+ */
+
+#include <stdlib.h>
+#include <stdio.h>
+#include <string.h>
+#include <assert.h>
+
+#include <sys/socket.h>
+#include <netinet/in.h>
+#include <arpa/inet.h>
+
+#include "../../lib/zdlibc/logger.h"
+#include "../../lib/zdlibc/redis.h"
+
+#include "output_modules.h"
+
+#define UNUSED __attribute__((unused))
+
+typedef struct scannable_t {
+	in_addr_t ip_address;
+	uint8_t source;
+} scannable_t;
+
+#define QUEUE_NAME "zmap_results"
+#define BUFFER_SIZE 500
+#define SOURCE_ZMAP 0
+
+static scannable_t* buffer;
+static int buffer_fill = 0;
+
+int redismodule_init(UNUSED struct state_conf *conf)
+{
+	buffer = calloc(BUFFER_SIZE, sizeof(scannable_t));
+	assert(buffer);
+	buffer_fill = 0;
+	return redis_init();
+}
+
+int redismodule_flush(void)
+{
+	if (redis_lpush(QUEUE_NAME, buffer,
+			buffer_fill, sizeof(scannable_t))) {
+		return EXIT_FAILURE;
+	}
+	buffer_fill = 0;
+	return EXIT_SUCCESS;
+}
+
+int redismodule_newip(ipaddr_n_t saddr, UNUSED ipaddr_n_t daddr,
+        UNUSED port_n_t sport, UNUSED port_n_t dport,
+        UNUSED const char *response_type, int is_repeat,
+        UNUSED int in_cooldown, UNUSED const u_char *packet)
+{
+	if (!is_repeat) {
+		buffer[buffer_fill].ip_address = saddr;
+		buffer[buffer_fill].source = SOURCE_ZMAP;
+
+		if (++buffer_fill == BUFFER_SIZE) {
+			if (redismodule_flush()) {
+				return EXIT_FAILURE;
+			}
+		}
+	}
+	return EXIT_SUCCESS;
+}
+
+int redismodule_close(UNUSED struct state_conf* c, 
+        UNUSED struct state_send* s,
+		UNUSED struct state_recv* r)
+{
+	if (redismodule_flush()) {
+		return EXIT_FAILURE;
+	}
+	if (redis_close()) {
+		return EXIT_FAILURE;
+	}
+	return EXIT_SUCCESS;
+}
+
+output_module_t module_redis = {
+	.name = "redis",
+	.init = &redismodule_init,
+	.start = NULL,
+	.update = NULL,
+	.update_interval = 0,
+	.close = &redismodule_close,
+	.success_ip = &redismodule_newip,
+	.other_ip = NULL
+};
+

src/output_modules/module_redis.h

@@ -0,0 +1,19 @@
+/*
+ * ZMap Copyright 2013 Regents of the University of Michigan 
+ * 
+ * Licensed under the Apache License, Version 2.0 (the "License"); you may not
+ * use this file except in compliance with the License. You may obtain a copy
+ * of the License at http://www.apache.org/licenses/LICENSE-2.0
+ */
+
+#include <output_modules.h>
+
+int redismodule_init(struct state_conf *conf);
+
+int redismodule_newip(ipaddr_n_t saddr, ipaddr_n_t daddr,
+        port_n_t sport, port_n_t dport, struct timeval* t,
+        const char *response_type, int is_repeat,
+        int in_cooldown, const u_char *packet);
+
+int redismodule_close(struct state_conf* c,
+		struct state_send* s, struct state_recv* r);

src/output_modules/module_simple_file.c

@@ -0,0 +1,75 @@
+/*
+ * ZMap Copyright 2013 Regents of the University of Michigan 
+ * 
+ * Licensed under the Apache License, Version 2.0 (the "License"); you may not
+ * use this file except in compliance with the License. You may obtain a copy
+ * of the License at http://www.apache.org/licenses/LICENSE-2.0
+ */
+
+
+#include <string.h>
+#include <stdio.h>
+#include <stdlib.h>
+#include <sys/socket.h>
+#include <netinet/in.h>
+#include <arpa/inet.h>
+#include <assert.h>
+#include <logger.h>
+
+#include "output_modules.h"
+
+#define UNUSED __attribute__((unused))
+
+static FILE *file = NULL;
+
+int simplefile_init(struct state_conf *conf)
+{
+	assert(conf);
+	if (conf->output_filename) {
+		if (!strcmp(conf->output_filename, "-")) {
+			file = stdout;
+		} else {
+			if (!(file = fopen(conf->output_filename, "w"))) {
+				perror("Couldn't open output file");
+				exit(EXIT_FAILURE);
+			}
+		}
+	}
+	return EXIT_SUCCESS;
+}
+
+int simplefile_synack_newip(ipaddr_n_t saddr, UNUSED ipaddr_n_t daddr,
+		UNUSED const char *response_type,
+		int is_repeat, UNUSED int in_cooldown, UNUSED const u_char *packet,
+		UNUSED size_t buflen)
+{	
+	if (file && !is_repeat) {
+		struct in_addr addr;
+		addr.s_addr = saddr;
+		fprintf(file, "%s\n", inet_ntoa(addr));
+	}
+	fflush(file);
+	return EXIT_SUCCESS;
+}
+
+int simplefile_close(UNUSED struct state_conf* c, 
+	UNUSED struct state_send* s, 
+	UNUSED struct state_recv* r)
+{
+	if (file) {
+		fflush(file);
+		fclose(file);
+	}
+	return EXIT_SUCCESS;
+}
+
+output_module_t module_simple_file = {
+	.name = "simple_file",
+	.init = &simplefile_init,
+	.start = NULL,
+	.update = NULL,
+	.update_interval = 0,
+	.close = &simplefile_close,
+	.success_ip = &simplefile_synack_newip,
+	.other_ip = NULL,
+};

src/output_modules/output_modules.c

@@ -0,0 +1,54 @@
+/*
+ * ZMap Copyright 2013 Regents of the University of Michigan 
+ * 
+ * Licensed under the Apache License, Version 2.0 (the "License"); you may not
+ * use this file except in compliance with the License. You may obtain a copy
+ * of the License at http://www.apache.org/licenses/LICENSE-2.0
+ */
+
+
+#include <stdio.h>
+#include <string.h>
+
+#include "output_modules.h"
+
+extern output_module_t module_simple_file;
+extern output_module_t module_extended_file;
+// ADD YOUR MODULE HERE
+
+#ifdef REDIS
+extern output_module_t module_redis;
+extern output_module_t module_ssldbfeed;
+#endif
+
+
+output_module_t* output_modules[] = {
+	&module_simple_file,
+	&module_extended_file,
+#ifdef REDIS
+	&module_redis,
+	&module_ssldbfeed,
+#endif
+	// ADD YOUR MODULE HERE
+};
+
+
+
+output_module_t* get_output_module_by_name(const char* name)
+{
+    int num_modules = (int) (sizeof(output_modules)/sizeof(output_modules[0]));
+	for (int i=0; i < num_modules; i++) {
+		if (!strcmp(output_modules[i]->name, name)) {
+			return output_modules[i];
+		}
+	}
+	return NULL;
+}
+
+void print_output_modules(void)
+{
+    int num_modules = (int) (sizeof(output_modules)/sizeof(output_modules[0]));
+	for (int i=0; i < num_modules; i++) {
+		printf("%s\n", output_modules[i]->name);
+	}
+}

src/output_modules/output_modules.h

@@ -0,0 +1,44 @@
+/*
+ * ZMap Copyright 2013 Regents of the University of Michigan 
+ * 
+ * Licensed under the Apache License, Version 2.0 (the "License"); you may not
+ * use this file except in compliance with the License. You may obtain a copy
+ * of the License at http://www.apache.org/licenses/LICENSE-2.0
+ */
+
+#ifndef HEADER_OUTPUT_MODULES_H
+#define HEADER_OUTPUT_MODULES_H
+
+#include "../state.h"
+
+// called at scanner initialization
+typedef int (*output_init_cb)(struct state_conf *);
+
+// called on packet receipt
+typedef int (*output_packet_cb)(ipaddr_n_t src_ip, ipaddr_n_t dst_ip,
+				const char* response_type,
+				int is_repeat, int in_cooldown,
+				const u_char* packetbuf, size_t buflen);
+
+// called periodically during the scan
+typedef int (*output_update_cb)(struct state_conf*, struct state_send*, struct state_recv*);
+
+
+typedef struct output_module {
+	const char *name;
+	unsigned update_interval;
+
+	output_init_cb init;
+	output_update_cb start;
+	output_update_cb update;
+	output_update_cb close;
+	output_packet_cb success_ip;
+	output_packet_cb other_ip;
+
+} output_module_t;
+
+
+output_module_t* get_output_module_by_name(const char*);
+void print_output_modules(void);
+
+#endif // HEADER_OUTPUT_MODULES_H

src/probe_modules/module_icmp_echo.c

@@ -0,0 +1,219 @@
+/*
+ * ZMap Copyright 2013 Regents of the University of Michigan 
+ * 
+ * Licensed under the Apache License, Version 2.0 (the "License"); you may not
+ * use this file except in compliance with the License. You may obtain a copy
+ * of the License at http://www.apache.org/licenses/LICENSE-2.0
+ */
+
+// probe module for performing ICMP echo request (ping) scans
+
+#include <stdlib.h>
+#include <stdio.h>
+#include <stdint.h>
+#include <time.h>
+#include <unistd.h>
+#include <string.h>
+
+#include <netinet/ether.h>
+#include <netinet/ip.h>
+#include <netinet/ip_icmp.h>
+
+#include <sys/socket.h>
+#include <netinet/in.h>
+#include <arpa/inet.h>
+
+#include "probe_modules.h"
+#include "packet.h"
+#include "validate.h"
+
+probe_module_t module_icmp_echo;
+
+int icmp_echo_init_perthread(void* buf, macaddr_t *src,
+		macaddr_t *gw, __attribute__((unused)) port_h_t dst_port)
+{
+	memset(buf, 0, MAX_PACKET_SIZE);
+
+	struct ethhdr *eth_header = (struct ethhdr *)buf;
+	make_eth_header(eth_header, src, gw);
+
+	struct iphdr *ip_header = (struct iphdr*)(&eth_header[1]);
+	uint16_t len = htons(sizeof(struct iphdr) + sizeof(struct icmp) - 8);
+	make_ip_header(ip_header, IPPROTO_ICMP, len);
+
+	struct icmp *icmp_header = (struct icmp*)(&ip_header[1]);
+	make_icmp_header(icmp_header);
+
+	return EXIT_SUCCESS;
+}
+
+int icmp_echo_make_packet(void *buf, ipaddr_n_t src_ip, ipaddr_n_t dst_ip,
+				uint32_t *validation, __attribute__((unused))int probe_num)
+{
+	struct ethhdr *eth_header = (struct ethhdr *)buf;
+	struct iphdr *ip_header = (struct iphdr*)(&eth_header[1]);
+	struct icmp *icmp_header = (struct icmp*)(&ip_header[1]);
+	uint16_t icmp_idnum = validation[2] & 0xFFFF;
+
+	ip_header->saddr = src_ip;
+	ip_header->daddr = dst_ip;
+
+	icmp_header->icmp_id = icmp_idnum;
+	icmp_header->icmp_cksum = 0;
+	icmp_header->icmp_cksum = icmp_checksum((unsigned short *) icmp_header);
+
+	ip_header->check = 0;
+	ip_header->check = ip_checksum((unsigned short *) ip_header);
+
+	return EXIT_SUCCESS;
+}
+
+void icmp_echo_print_packet(FILE *fp, void* packet)
+{
+	struct ethhdr *ethh = (struct ethhdr *) packet;
+	struct iphdr *iph = (struct iphdr *) &ethh[1];
+	struct icmp *icmp_header = (struct icmp*)(&iph[1]);
+
+	fprintf(fp, "icmp { type: %u | code: %u "
+			"| checksum: %u | id: %u | seq: %u }\n",
+			icmp_header->icmp_type,
+			icmp_header->icmp_code,
+			ntohs(icmp_header->icmp_cksum),
+			ntohs(icmp_header->icmp_id),
+			ntohs(icmp_header->icmp_seq));
+	struct in_addr *s = (struct in_addr *) &(iph->saddr);
+	struct in_addr *d = (struct in_addr *) &(iph->daddr);
+	char srcip[20];
+	char dstip[20];
+	// inet_ntoa is a const char * so we if just call it in
+	// fprintf, you'll get back wrong results since we're
+	// calling it twice.
+	strncpy(srcip, inet_ntoa(*s), 19);
+	strncpy(dstip, inet_ntoa(*d), 19);
+	fprintf(fp, "ip { saddr: %s | daddr: %s | checksum: %u }\n",
+			srcip,
+			dstip,
+			ntohl(iph->check));
+	fprintf(fp, "eth { shost: %02x:%02x:%02x:%02x:%02x:%02x | "
+			"dhost: %02x:%02x:%02x:%02x:%02x:%02x }\n",
+			(int) ((unsigned char *) ethh->h_source)[0],
+			(int) ((unsigned char *) ethh->h_source)[1],
+			(int) ((unsigned char *) ethh->h_source)[2],
+			(int) ((unsigned char *) ethh->h_source)[3],
+			(int) ((unsigned char *) ethh->h_source)[4],
+			(int) ((unsigned char *) ethh->h_source)[5],
+			(int) ((unsigned char *) ethh->h_dest)[0],
+			(int) ((unsigned char *) ethh->h_dest)[1],
+			(int) ((unsigned char *) ethh->h_dest)[2],
+			(int) ((unsigned char *) ethh->h_dest)[3],
+			(int) ((unsigned char *) ethh->h_dest)[4],
+			(int) ((unsigned char *) ethh->h_dest)[5]);
+	fprintf(fp, "------------------------------------------------------\n");
+}
+
+response_type_t* icmp_echo_classify_packet(const u_char *packet, uint32_t len)
+{
+	(void)len;
+	struct iphdr *ip_hdr = (struct iphdr *)&packet[sizeof(struct ethhdr)];
+	struct icmp *icmp_hdr = (struct icmp*)((char *)ip_hdr
+					+ sizeof(struct iphdr)); 
+	switch (icmp_hdr->icmp_type) {
+		case ICMP_ECHOREPLY:
+			return &(module_icmp_echo.responses[0]);
+		case ICMP_UNREACH: 
+			return &(module_icmp_echo.responses[1]);
+		case ICMP_SOURCEQUENCH:
+			return &(module_icmp_echo.responses[2]);
+		case ICMP_REDIRECT:
+			return &(module_icmp_echo.responses[3]);
+		case ICMP_TIMXCEED:
+			return &(module_icmp_echo.responses[4]);
+		default:
+			return &(module_icmp_echo.responses[5]);
+	}
+}
+
+int icmp_validate_packet(const struct iphdr *ip_hdr, uint32_t len, uint32_t *src_ip, uint32_t *validation)
+{
+	if (ip_hdr->protocol != IPPROTO_ICMP) {
+		return 0;
+	}
+	
+	if ((4*ip_hdr->ihl + sizeof(struct icmphdr)) > len) {
+		// buffer not large enough to contain expected icmp header 
+		return 0;
+	}
+	
+	struct icmphdr *icmp_h = (struct icmphdr*)((char *)ip_hdr + 4*ip_hdr->ihl);
+	uint16_t icmp_idnum = icmp_h->un.echo.id;
+
+	// ICMP validation is tricky: for some packet types, we must look inside 
+	// the payload
+	if (icmp_h->type == ICMP_TIME_EXCEEDED || icmp_h->type == ICMP_DEST_UNREACH) {
+		if ((4*ip_hdr->ihl + sizeof(struct icmphdr) +
+			sizeof(struct iphdr)) > len) {
+			return 0;
+		}
+		struct iphdr *ip_inner = (struct iphdr *)(icmp_h + 1);
+		if ((4*ip_hdr->ihl + sizeof(struct icmphdr) +
+				4*ip_inner->ihl + sizeof(struct icmphdr)) > len) {
+			return 0;
+		}
+		struct icmphdr *icmp_inner = (struct icmphdr*)((char *)ip_inner + 4 *ip_hdr->ihl);
+
+		// Regenerate validation and icmp id based off inner payload
+		icmp_idnum = icmp_inner->un.echo.id;
+		*src_ip = ip_inner->daddr;
+		validate_gen(ip_hdr->daddr, ip_inner->daddr, (uint8_t *)validation);
+	} 
+
+	// validate icmp id
+	if (icmp_idnum != (validation[2] & 0xFFFF)) {
+		return 0;
+	}
+
+	return 1;
+}
+
+static response_type_t responses[] = {
+	{
+		.name = "echoreply",
+		.is_success = 1	
+	},
+	{
+		.name = "unreach",
+		.is_success = 0
+	},
+	{
+		.name = "sourcequench",
+		.is_success = 0
+	},
+	{
+		.name = "redirect",
+		.is_success = 0
+	},
+	{
+		.name = "timxceed",
+		.is_success = 0
+	},
+	{
+		.name = "other",
+		.is_success = 0
+	}
+};
+
+probe_module_t module_icmp_echo = {
+	.name = "icmp_echoscan",
+	.packet_length = 62,
+	.pcap_filter = "icmp and icmp[0]!=8",
+	.pcap_snaplen = 96,
+	.port_args = 0,
+	.thread_initialize = &icmp_echo_init_perthread,
+	.make_packet = &icmp_echo_make_packet,
+	.print_packet = &icmp_echo_print_packet,
+	.classify_packet = &icmp_echo_classify_packet,
+	.validate_packet = &icmp_validate_packet,
+	.close = NULL,
+	.responses = responses
+};
+

src/probe_modules/module_tcp_synscan.c

@@ -0,0 +1,196 @@
+/*
+ * ZMap Copyright 2013 Regents of the University of Michigan 
+ * 
+ * Licensed under the Apache License, Version 2.0 (the "License"); you may not
+ * use this file except in compliance with the License. You may obtain a copy
+ * of the License at http://www.apache.org/licenses/LICENSE-2.0
+ */
+
+// probe module for performing TCP SYN scans 
+
+#include <stdlib.h>
+#include <stdio.h>
+#include <stdint.h>
+#include <unistd.h>
+#include <string.h>
+
+#include <netinet/ip.h>
+#include <netinet/tcp.h>
+#include <netinet/ether.h>
+
+#include <sys/socket.h>
+#include <netinet/in.h>
+#include <arpa/inet.h>
+
+#include "probe_modules.h"
+#include "packet.h"
+
+probe_module_t module_tcp_synscan;
+uint32_t num_ports = 1;
+
+int synscan_init_perthread(void* buf, macaddr_t *src,
+		macaddr_t *gw, port_h_t dst_port)
+{
+	memset(buf, 0, MAX_PACKET_SIZE);
+	struct ethhdr *eth_header = (struct ethhdr *)buf;
+	make_eth_header(eth_header, src, gw);
+	struct iphdr *ip_header = (struct iphdr*)(&eth_header[1]);
+	uint16_t len = htons(sizeof(struct iphdr) + sizeof(struct tcphdr));
+	make_ip_header(ip_header, IPPROTO_TCP, len);
+	struct tcphdr *tcp_header = (struct tcphdr*)(&ip_header[1]);
+	make_tcp_header(tcp_header, dst_port);
+	num_ports = zconf.source_port_last - zconf.source_port_first + 1;
+	return EXIT_SUCCESS;
+}
+
+int synscan_make_packet(void *buf, ipaddr_n_t src_ip, ipaddr_n_t dst_ip,
+		uint32_t *validation, int probe_num)
+{
+	struct ethhdr *eth_header = (struct ethhdr *)buf;
+	struct iphdr *ip_header = (struct iphdr*)(&eth_header[1]);
+	struct tcphdr *tcp_header = (struct tcphdr*)(&ip_header[1]);
+	uint16_t src_port = zconf.source_port_first 
+					+ ((validation[1] + probe_num) % num_ports);
+	uint32_t tcp_seq = validation[0];
+	
+
+	ip_header->saddr = src_ip;
+	ip_header->daddr = dst_ip;
+
+	tcp_header->source = htons(src_port);
+	tcp_header->seq = tcp_seq;
+	tcp_header->check = 0;
+	tcp_header->check = tcp_checksum(sizeof(struct tcphdr),
+			ip_header->saddr, ip_header->daddr, tcp_header);
+
+	ip_header->check = 0;
+	ip_header->check = ip_checksum((unsigned short *) ip_header);
+
+	return EXIT_SUCCESS;
+}
+
+void synscan_print_packet(FILE *fp, void* packet)
+{
+	struct ethhdr *ethh = (struct ethhdr *) packet;
+	struct iphdr *iph = (struct iphdr *) &ethh[1];
+	struct tcphdr *tcph = (struct tcphdr *) &iph[1];
+	fprintf(fp, "tcp { source: %u | dest: %u | seq: %u | checksum: %u }\n",
+			ntohs(tcph->source),
+			ntohs(tcph->dest),
+			ntohl(tcph->seq),
+			ntohl(tcph->check));
+	struct in_addr *s = (struct in_addr *) &(iph->saddr);
+	struct in_addr *d = (struct in_addr *) &(iph->daddr);
+	char srcip[20];
+	char dstip[20];
+	// inet_ntoa is a const char * so we if just call it in
+	// fprintf, you'll get back wrong results since we're
+	// calling it twice.
+	strncpy(srcip, inet_ntoa(*s), 19);
+	strncpy(dstip, inet_ntoa(*d), 19);
+	fprintf(fp, "ip { saddr: %s | daddr: %s | checksum: %u }\n",
+			srcip,
+			dstip,
+			ntohl(iph->check));
+	fprintf(fp, "eth { shost: %02x:%02x:%02x:%02x:%02x:%02x | "
+			"dhost: %02x:%02x:%02x:%02x:%02x:%02x }\n",
+			(int) ((unsigned char *) ethh->h_source)[0],
+			(int) ((unsigned char *) ethh->h_source)[1],
+			(int) ((unsigned char *) ethh->h_source)[2],
+			(int) ((unsigned char *) ethh->h_source)[3],
+			(int) ((unsigned char *) ethh->h_source)[4],
+			(int) ((unsigned char *) ethh->h_source)[5],
+			(int) ((unsigned char *) ethh->h_dest)[0],
+			(int) ((unsigned char *) ethh->h_dest)[1],
+			(int) ((unsigned char *) ethh->h_dest)[2],
+			(int) ((unsigned char *) ethh->h_dest)[3],
+			(int) ((unsigned char *) ethh->h_dest)[4],
+			(int) ((unsigned char *) ethh->h_dest)[5]);
+	fprintf(fp, "------------------------------------------------------\n");
+}
+
+response_type_t* synscan_classify_packet(const u_char *packet, uint32_t len)
+{
+	(void)len;
+	struct iphdr *ip_hdr = (struct iphdr *)&packet[sizeof(struct ethhdr)];
+	struct tcphdr *tcp = (struct tcphdr*)((char *)ip_hdr 
+					+ (sizeof(struct iphdr)));
+	if (tcp->rst) { // RST packet
+			return &(module_tcp_synscan.responses[1]);
+	} else { // SYNACK packet
+			return &(module_tcp_synscan.responses[0]);
+	}
+}
+// Returns 0 if dst_port is outside the expected valid range, non-zero otherwise
+static inline int check_dst_port(uint16_t port, uint32_t *validation)
+{
+	if (port > zconf.source_port_last 
+					|| port < zconf.source_port_first) {
+		return EXIT_FAILURE;
+	}
+	int32_t to_validate = port - zconf.source_port_first;
+	int32_t min = validation[1] % num_ports;
+	int32_t max = (validation[1] + zconf.packet_streams - 1) % num_ports;
+
+	return (((max - min) % num_ports) >= ((to_validate - min) % num_ports));
+}
+
+int synscan_validate_packet(const struct iphdr *ip_hdr, uint32_t len, 
+				__attribute__((unused))uint32_t *src_ip, uint32_t *validation)
+{
+	if (ip_hdr->protocol != IPPROTO_TCP) {
+		return 0;
+	}
+
+	if ((4*ip_hdr->ihl + sizeof(struct tcphdr)) > len) {
+		// buffer not large enough to contain expected tcp header 
+		return 0;
+	}
+	struct tcphdr *tcp = (struct tcphdr*)((char *)ip_hdr + 4*ip_hdr->ihl);
+	uint16_t sport = tcp->source;
+	uint16_t dport = tcp->dest;
+
+	// validate source port
+	if (ntohs(sport) != zconf.target_port) {
+		return 0;
+	}
+
+	// validate destination port
+	if (!check_dst_port(ntohs(dport), validation)) {
+		return 0;
+	}
+
+	// validate tcp acknowledgement number
+	if (htonl(tcp->ack_seq) != htonl(validation[0])+1) {
+		return 0;
+	}
+	
+	return 1;
+}
+
+static response_type_t responses[] = {
+	{
+		.is_success = 1,	
+		.name = "synack"
+	},
+	{
+		.is_success = 0,
+		.name = "rst"
+	}
+};
+
+probe_module_t module_tcp_synscan = {
+	.name = "tcp_synscan",
+	.packet_length = 54,
+	.pcap_filter = "tcp && tcp[13] & 4 != 0 || tcp[13] == 18",
+	.pcap_snaplen = 96,
+	.port_args = 1,
+	.thread_initialize = &synscan_init_perthread,
+	.make_packet = &synscan_make_packet,
+	.print_packet = &synscan_print_packet,
+	.classify_packet = &synscan_classify_packet,
+	.validate_packet = &synscan_validate_packet,
+	.close = NULL,
+	.responses = responses,
+};
+

src/probe_modules/module_udp.c

@@ -0,0 +1,227 @@
+/*
+ * ZMap Copyright 2013 Regents of the University of Michigan 
+ * 
+ * Licensed under the Apache License, Version 2.0 (the "License"); you may not
+ * use this file except in compliance with the License. You may obtain a copy
+ * of the License at http://www.apache.org/licenses/LICENSE-2.0
+ */
+
+/* send module for performing TCP SYN scans */
+
+#include <stdlib.h>
+#include <stdio.h>
+#include <stdint.h>
+#include <unistd.h>
+#include <string.h>
+#include <assert.h>
+
+#include <netinet/udp.h>
+#include <netinet/ip.h>
+#include <netinet/ether.h>
+
+#include <sys/socket.h>
+#include <netinet/in.h>
+#include <arpa/inet.h>
+
+#include "probe_modules.h"
+#include "packet.h"
+
+const char *udp_send_msg = "GET / HTTP/1.1\r\n\r\n"; // Must be null-terminated
+static int num_ports = 1;
+
+probe_module_t module_udp;
+
+int udp_init_perthread(void* buf, macaddr_t *src,
+		macaddr_t *gw, __attribute__((unused)) port_h_t dst_port)
+{
+	memset(buf, 0, MAX_PACKET_SIZE);
+	struct ethhdr *eth_header = (struct ethhdr *)buf;
+	make_eth_header(eth_header, src, gw);
+	struct iphdr *ip_header = (struct iphdr*)(&eth_header[1]);
+	uint16_t len = htons(sizeof(struct iphdr) + sizeof(struct udphdr) + strlen(udp_send_msg));
+	make_ip_header(ip_header, IPPROTO_UDP, len);
+
+	struct udphdr *udp_header = (struct udphdr*)(&ip_header[1]);
+	len = sizeof(struct udphdr) + strlen(udp_send_msg);
+	make_udp_header(udp_header, zconf.target_port, len);
+
+	char* payload = (char*)(&udp_header[1]);
+
+	module_udp.packet_length = sizeof(struct ethhdr) + sizeof(struct iphdr) 
+				+ sizeof(struct udphdr) + strlen(udp_send_msg);
+	assert(module_udp.packet_length <= MAX_PACKET_SIZE);
+
+	strcpy(payload, udp_send_msg);
+
+	num_ports = zconf.source_port_last - zconf.source_port_first + 1;
+
+	return EXIT_SUCCESS;
+}
+
+int udp_make_packet(void *buf, ipaddr_n_t src_ip, ipaddr_n_t dst_ip, 
+		uint32_t *validation, int probe_num)
+{
+	struct ethhdr *eth_header = (struct ethhdr *)buf;
+	struct iphdr *ip_header = (struct iphdr*)(&eth_header[1]);
+	struct udphdr *udp_header = (struct udphdr*)(&ip_header[1]);
+	uint16_t src_port = zconf.source_port_first
+					+ ((validation[1] + probe_num) % num_ports);
+
+	ip_header->saddr = src_ip;
+	ip_header->daddr = dst_ip;
+	udp_header->source = src_port;
+
+	ip_header->check = 0;
+	ip_header->check = ip_checksum((unsigned short *) ip_header);
+
+	return EXIT_SUCCESS;
+}
+
+void udp_print_packet(FILE *fp, void* packet)
+{
+	struct ethhdr *ethh = (struct ethhdr *) packet;
+	struct iphdr *iph = (struct iphdr *) &ethh[1];
+	struct udphdr *udph  = (struct udphdr*)(&iph[1]);
+	fprintf(fp, "udp { source: %u | dest: %u | checksum: %u }\n",
+			ntohs(udph->source),
+			ntohs(udph->dest),
+			ntohl(udph->check));
+	//ip_header = (struct iphdr*)(&eth_header[1])
+	struct in_addr *s = (struct in_addr *) &(iph->saddr);
+	struct in_addr *d = (struct in_addr *) &(iph->daddr);
+	char srcip[20];
+	char dstip[20];
+	// inet_ntoa is a const char * so we if just call it in
+	// fprintf, you'll get back wrong results since we're
+	// calling it twice.
+	strncpy(srcip, inet_ntoa(*s), 19);
+	strncpy(dstip, inet_ntoa(*d), 19);
+	fprintf(fp, "ip { saddr: %s | daddr: %s | checksum: %u }\n",
+			srcip,
+			dstip,
+			ntohl(iph->check));
+	fprintf(fp, "eth { shost: %02x:%02x:%02x:%02x:%02x:%02x | "
+			"dhost: %02x:%02x:%02x:%02x:%02x:%02x }\n",
+			(int) ((unsigned char *) ethh->h_source)[0],
+			(int) ((unsigned char *) ethh->h_source)[1],
+			(int) ((unsigned char *) ethh->h_source)[2],
+			(int) ((unsigned char *) ethh->h_source)[3],
+			(int) ((unsigned char *) ethh->h_source)[4],
+			(int) ((unsigned char *) ethh->h_source)[5],
+			(int) ((unsigned char *) ethh->h_dest)[0],
+			(int) ((unsigned char *) ethh->h_dest)[1],
+			(int) ((unsigned char *) ethh->h_dest)[2],
+			(int) ((unsigned char *) ethh->h_dest)[3],
+			(int) ((unsigned char *) ethh->h_dest)[4],
+			(int) ((unsigned char *) ethh->h_dest)[5]);
+	fprintf(fp, "------------------------------------------------------\n");
+}
+
+
+response_type_t* udp_classify_packet(const u_char *packet, uint32_t len)
+{
+	(void)len;
+	struct iphdr *ip_hdr = (struct iphdr *)&packet[sizeof(struct ethhdr)];
+	if (ip_hdr->protocol == IPPROTO_UDP) {
+		return &(module_udp.responses[0]);
+	} else if (ip_hdr->protocol == IPPROTO_ICMP) {
+		return &(module_udp.responses[1]);
+	} else {
+		return &(module_udp.responses[2]);
+	}
+}
+
+// Returns 0 if dst_port is outside the expected valid range, non-zero otherwise
+static inline int check_dst_port(uint16_t port, uint32_t *validation)
+{
+	if (port > zconf.source_port_last 
+					|| port < zconf.source_port_first) {
+		return EXIT_FAILURE;
+	}
+	int32_t to_validate = port - zconf.source_port_first;
+	int32_t min = validation[1] % num_ports;
+	int32_t max = (validation[1] + zconf.packet_streams - 1) % num_ports;
+
+	return (((max - min) % num_ports) >= ((to_validate - min) % num_ports));
+}
+
+int udp_validate_packet(const struct iphdr *ip_hdr, uint32_t len, 
+		__attribute__((unused))uint32_t *src_ip, uint32_t *validation)
+{
+	uint16_t dport, sport;
+	if (ip_hdr->protocol == IPPROTO_UDP) {
+		if ((4*ip_hdr->ihl + sizeof(struct udphdr)) > len) {
+			// buffer not large enough to contain expected udp header 
+			return 0;
+		}
+		struct udphdr *udp = (struct udphdr*)((char *)ip_hdr + 4*ip_hdr->ihl);
+
+		sport = ntohs(udp->dest);
+		dport = ntohs(udp->source);
+	} else if (ip_hdr->protocol == IPPROTO_ICMP) {
+		// UDP can return ICMP Destination unreach
+		// IP( ICMP( IP( UDP ) ) ) for a destination unreach
+		uint32_t min_len = 4*ip_hdr->ihl + sizeof(struct icmphdr)
+				+ sizeof(struct iphdr) + sizeof(struct udphdr);
+		if (len < min_len) {
+			// Not enough information for us to validate
+			return 0;
+		}
+
+		struct icmphdr *icmp = (struct icmphdr*)((char *)ip_hdr + 4*ip_hdr->ihl);
+		if (icmp->type != ICMP_DEST_UNREACH) {
+			return 0;
+		}
+		
+		struct iphdr *ip_inner = (struct iphdr*)&icmp[1];
+		// Now we know the actual inner ip length, we should recheck the buffer
+		if (len < 4*ip_inner->ihl - sizeof(struct iphdr) + min_len) {
+			return 0;
+		}
+		// This is the packet we sent
+		struct udphdr *udp = (struct udphdr *)((char*)ip_inner + 4*ip_inner->ihl);
+
+		sport = ntohs(udp->source);
+		dport = ntohs(udp->dest);	
+	} else {
+		return 0;
+	}
+	if (dport != zconf.target_port) {
+		return 0;
+	}
+	if (!check_dst_port(sport, validation)) {
+		return 0;
+	}
+	return 1;
+}
+
+static response_type_t responses[] = {
+	{
+		.is_success = 1,
+		.name = "data"
+	},
+	{
+		.is_success = 0,
+		.name = "port-unreach"
+	},
+	{
+		.is_success = 0,
+		.name = "invalid"
+	}
+};
+
+probe_module_t module_udp = {
+	.name = "udp",
+	.packet_length = 96,
+	.pcap_filter = "udp || icmp",
+	.pcap_snaplen = 96,
+	.port_args = 1,
+	.thread_initialize = &udp_init_perthread,
+	.make_packet = &udp_make_packet,
+	.print_packet = &udp_print_packet,
+	.validate_packet = &udp_validate_packet,
+	.classify_packet = &udp_classify_packet,
+	.close = NULL,
+	.responses = responses
+};
+

src/probe_modules/packet.c

@@ -0,0 +1,89 @@
+/*
+ * ZMap Copyright 2013 Regents of the University of Michigan 
+ * 
+ * Licensed under the Apache License, Version 2.0 (the "License"); you may not
+ * use this file except in compliance with the License. You may obtain a copy
+ * of the License at http://www.apache.org/licenses/LICENSE-2.0
+ */
+
+
+#include "packet.h"
+
+#include <stdlib.h>
+#include <stdio.h>
+#include <string.h>
+
+#include <netinet/ether.h>
+#include <netinet/ip.h>
+#include <netinet/tcp.h>
+#include <netinet/udp.h>
+#include <netinet/ip_icmp.h>
+
+#include <net/if.h>
+#include <arpa/inet.h>
+
+#include "state.h"
+
+void print_macaddr(struct ifreq* i)
+{
+	printf("Device %s -> Ethernet %02x:%02x:%02x:%02x:%02x:%02x\n",
+			i->ifr_name,
+			(int) ((unsigned char *) &i->ifr_hwaddr.sa_data)[0],
+			(int) ((unsigned char *) &i->ifr_hwaddr.sa_data)[1],
+			(int) ((unsigned char *) &i->ifr_hwaddr.sa_data)[2],
+			(int) ((unsigned char *) &i->ifr_hwaddr.sa_data)[3],
+			(int) ((unsigned char *) &i->ifr_hwaddr.sa_data)[4],
+			(int) ((unsigned char *) &i->ifr_hwaddr.sa_data)[5]);
+}
+
+void make_eth_header(struct ethhdr *ethh, macaddr_t *src, macaddr_t *dst)
+{
+	memcpy(ethh->h_source, src, ETH_ALEN);
+	memcpy(ethh->h_dest, dst, ETH_ALEN);
+	ethh->h_proto = htons(ETH_P_IP);
+}
+
+void make_ip_header(struct iphdr *iph, uint8_t protocol, uint16_t len)
+{	   
+	iph->ihl = 5; // Internet Header Length
+	iph->version = 4; // IPv4
+	iph->tos = 0; // Type of Service
+	iph->tot_len = len; 
+	iph->id = htons(54321); // identification number
+	iph->frag_off = 0; //fragmentation falg
+	iph->ttl = MAXTTL; // time to live (TTL)
+	iph->protocol = protocol; // upper layer protocol => TCP
+	// we set the checksum = 0 for now because that's
+	// what it needs to be when we run the IP checksum
+	iph->check = 0;
+}
+
+void make_icmp_header(struct icmp *buf)
+{
+	buf->icmp_type = ICMP_ECHO;
+	buf->icmp_code = 0;
+	buf->icmp_seq = 0;
+}
+
+void make_tcp_header(struct tcphdr *tcp_header, port_h_t dest_port)
+{
+    tcp_header->seq = random();
+    tcp_header->ack_seq = 0;
+    tcp_header->res2 = 0;
+    tcp_header->doff = 5; // data offset 
+    tcp_header->syn = 1;
+    tcp_header->window = htons(65535); // largest possible window
+    tcp_header->check = 0;
+    tcp_header->urg_ptr = 0;
+    tcp_header->dest = htons(dest_port);
+}
+
+void make_udp_header(struct udphdr *udp_header, port_h_t dest_port,
+				uint16_t len)
+{
+	udp_header->dest = htons(dest_port);
+	udp_header->len = htons(len);
+	// checksum ignored in IPv4 if 0
+	udp_header->check = 0;
+}
+

src/probe_modules/packet.h

@@ -0,0 +1,82 @@
+#include "state.h"
+
+#include <netinet/ether.h>
+#include <netinet/ip.h>
+#include <netinet/tcp.h>
+#include <netinet/ip_icmp.h>
+#include <netinet/udp.h>
+
+#ifndef HEADER_ZMAP_PACKET_H
+#define HEADER_ZMAP_PACKET_H
+
+#define MAX_PACKET_SIZE 4096
+
+typedef unsigned short __attribute__((__may_alias__)) alias_unsigned_short;
+
+void make_eth_header(struct ethhdr *ethh, macaddr_t *src, macaddr_t *dst);
+
+void make_ip_header(struct iphdr *iph, uint8_t, uint16_t);
+void make_tcp_header(struct tcphdr*, port_h_t);
+void make_icmp_header(struct icmp *);
+void make_udp_header(struct udphdr *udp_header, port_h_t dest_port,
+				uint16_t len);
+
+static inline unsigned short in_checksum(unsigned short *ip_pkt, int len)
+{
+	unsigned long sum = 0;
+	for (int nwords = len/2; nwords > 0; nwords--) {
+		sum += *ip_pkt++;
+	}
+	sum = (sum >> 16) + (sum & 0xffff);
+	sum += (sum >> 16);
+	return (unsigned short) (~sum);
+}
+
+__attribute__((unused)) static inline unsigned short ip_checksum(
+                unsigned short *buf)
+{
+	return in_checksum(buf, (int) sizeof(struct iphdr));
+}
+
+__attribute__((unused)) static inline unsigned short icmp_checksum(
+                unsigned short *buf)
+{
+	return in_checksum(buf, (int) sizeof(struct icmp));
+}
+
+static __attribute__((unused)) uint16_t tcp_checksum(unsigned short len_tcp,
+		uint32_t saddr, uint32_t daddr, struct tcphdr *tcp_pkt)
+{
+	alias_unsigned_short *src_addr = (alias_unsigned_short *) &saddr;
+	alias_unsigned_short *dest_addr = (alias_unsigned_short *) &daddr;
+
+	unsigned char prot_tcp = 6;
+	unsigned long sum = 0;
+	int nleft = len_tcp;
+	unsigned short *w;
+
+	w = (unsigned short *) tcp_pkt;
+	// calculate the checksum for the tcp header and tcp data
+	while(nleft > 1) {
+		sum += *w++;
+		nleft -= 2;
+	}
+	// if nleft is 1 there ist still on byte left.
+	// We add a padding byte (0xFF) to build a 16bit word
+	if (nleft > 0) {
+		sum += *w & ntohs(0xFF00);
+	}
+	// add the pseudo header
+	sum += src_addr[0];
+	sum += src_addr[1];
+	sum += dest_addr[0];
+	sum += dest_addr[1];
+	sum += htons(len_tcp);
+	sum += htons(prot_tcp);
+	sum = (sum >> 16) + (sum & 0xFFFF);
+	sum += (sum >> 16);
+	// Take the one's complement of sum
+	return (unsigned short) (~sum);
+}
+
+#endif

src/probe_modules/probe_modules.c

@@ -0,0 +1,42 @@
+/*
+ * ZMap Copyright 2013 Regents of the University of Michigan 
+ * 
+ * Licensed under the Apache License, Version 2.0 (the "License"); you may not
+ * use this file except in compliance with the License. You may obtain a copy
+ * of the License at http://www.apache.org/licenses/LICENSE-2.0
+ */
+
+#include <stdio.h>
+#include <string.h>
+
+#include "probe_modules.h"
+
+extern probe_module_t module_tcp_synscan;
+extern probe_module_t module_icmp_echo;
+extern probe_module_t module_udp;
+// ADD YOUR MODULE HERE
+
+probe_module_t* probe_modules[] = {
+	&module_tcp_synscan,
+	&module_icmp_echo,
+	&module_udp
+	// ADD YOUR MODULE HERE
+};
+
+
+probe_module_t* get_probe_module_by_name(const char* name)
+{
+	for (int i=0; i < (int) (sizeof(probe_modules)/sizeof(probe_modules[0])); i++) {
+		if (!strcmp(probe_modules[i]->name, name)) {
+			return probe_modules[i];
+		}
+	}
+	return NULL;
+}
+
+void print_probe_modules(void)
+{
+	for (int i=0; i < (int) (sizeof(probe_modules)/sizeof(probe_modules[0])); i++) {
+		printf("%s\n", probe_modules[i]->name);
+	}
+}

src/probe_modules/probe_modules.h

@@ -0,0 +1,45 @@
+#include "../state.h"
+
+#ifndef HEADER_PROBE_MODULES_H
+#define HEADER_PROBE_MODULES_H
+
+typedef struct probe_response_type {
+		const uint8_t is_success;
+		const char *name;
+} response_type_t;
+
+typedef int (*probe_global_init_cb)(struct state_conf *);
+typedef int (*probe_thread_init_cb)(void* packetbuf, macaddr_t* src_mac, macaddr_t* gw_mac, port_n_t src_port);
+typedef int (*probe_make_packet_cb)(void* packetbuf, ipaddr_n_t src_ip, ipaddr_n_t dst_ip,
+			uint32_t *validation, int probe_num);
+typedef void (*probe_print_packet_cb)(FILE *, void* packetbuf);
+typedef int (*probe_close_cb)(struct state_conf*,  struct state_send*, struct state_recv*);
+typedef response_type_t* (*probe_classify_packet_cb)(const u_char* packetbuf, uint32_t len);
+typedef int (*probe_validate_packet_cb)(const struct iphdr *ip_hdr, uint32_t len, uint32_t *src_ip, uint32_t *validation);
+
+typedef struct probe_module {
+	const char *name;
+	size_t packet_length;
+	const char *pcap_filter;
+	size_t pcap_snaplen;
+
+	// Should ZMap complain if the user hasn't specified valid
+	// source and target port numbers?
+	uint8_t port_args;
+
+	response_type_t *responses;
+
+	probe_global_init_cb global_initialize;
+	probe_thread_init_cb thread_initialize;
+	probe_make_packet_cb make_packet;
+	probe_print_packet_cb print_packet;
+	probe_validate_packet_cb validate_packet;
+	probe_classify_packet_cb classify_packet;
+	probe_close_cb close;
+
+} probe_module_t;
+
+probe_module_t* get_probe_module_by_name(const char*);
+void print_probe_modules(void);
+
+#endif // HEADER_PROBE_MODULES_H

src/recv.c

@@ -0,0 +1,195 @@
+/*
+ * ZMap Copyright 2013 Regents of the University of Michigan 
+ * 
+ * Licensed under the Apache License, Version 2.0 (the "License"); you may not
+ * use this file except in compliance with the License. You may obtain a copy
+ * of the License at http://www.apache.org/licenses/LICENSE-2.0
+ */
+
+#include <stdlib.h>
+#include <stdio.h>
+#include <string.h>
+#include <unistd.h>
+#include <pthread.h>
+
+#include <pcap.h>
+#include <pcap/pcap.h>
+
+#include <netinet/tcp.h>
+#include <netinet/ip_icmp.h>
+#include <netinet/udp.h>
+#include <netinet/ip.h>
+#include <netinet/in.h>
+
+#include <linux/if_ether.h>
+#include <arpa/inet.h>
+#include <sys/types.h>
+#include <sys/socket.h>
+#include <sys/ioctl.h>
+#include <net/if.h>
+
+#include "../lib/logger.h"
+
+#include "state.h"
+#include "validate.h"
+#include "probe_modules/probe_modules.h"
+#include "output_modules/output_modules.h"
+
+#define PCAP_PROMISC 1
+#define PCAP_TIMEOUT 1000
+
+static uint32_t num_src_ports;
+static pcap_t *pc = NULL;
+
+// bitmap of observed IP addresses
+static uint64_t *ip_seen = NULL;
+static const int IP_SEEN_SIZE = 0x4000000; // == 2^32/64
+
+// check if we've received a response from this address previously
+static inline int check_ip(uint32_t ip)
+{
+	return (ip_seen[ip >> 6] >> (ip & 0x3F)) & 1;
+}
+
+// set that we've received a response from the address
+static inline void set_ip(uint32_t ip)
+{
+	ip_seen[ip >> 6] |= (uint64_t)1 << (ip & 0x3F);
+}
+
+void packet_cb(u_char __attribute__((__unused__)) *user,
+		const struct pcap_pkthdr *p, const u_char *bytes)
+{
+	if (!p) {
+		return;
+	}
+	if (zrecv.success_unique >= zconf.max_results) {
+		// Libpcap can process multiple packets per pcap_dispatch;
+		// we need to throw out results once we've
+		// gotten our --max-results worth.
+		return;
+	}
+	// length of entire packet captured by libpcap
+	uint32_t buflen = (uint32_t) p->caplen;
+
+	if ((sizeof(struct iphdr) + sizeof(struct ethhdr)) > buflen) {
+		// buffer not large enough to contain ethernet
+		// and ip headers. further action would overrun buf
+		return;
+	}
+	struct iphdr *ip_hdr = (struct iphdr *)&bytes[sizeof(struct ethhdr)];
+	
+	uint32_t src_ip = ip_hdr->saddr;
+
+	uint32_t validation[VALIDATE_BYTES/sizeof(uint8_t)];
+	// TODO: for TTL exceeded messages, ip_hdr->saddr is going to be different
+	// and we must calculate off potential payload message instead
+	validate_gen(ip_hdr->daddr, ip_hdr->saddr, (uint8_t *)validation);
+
+	if (!zconf.probe_module->validate_packet(ip_hdr, buflen - sizeof(struct ethhdr),
+				&src_ip, validation)) {
+		return;
+	}
+
+	int is_repeat = check_ip(src_ip);
+	response_type_t *r = zconf.probe_module->classify_packet(bytes, buflen);
+
+	if (r->is_success) {
+		zrecv.success_total++;
+		if (!is_repeat) {
+			zrecv.success_unique++;
+			set_ip(src_ip);
+		}
+		if (zsend.complete) { 
+			zrecv.cooldown_total++;
+			if (!is_repeat) {
+				zrecv.cooldown_unique++;
+			}
+		}
+		if (zconf.output_module && zconf.output_module->success_ip) {
+			zconf.output_module->success_ip(
+					ip_hdr->saddr, ip_hdr->daddr,
+					r->name, is_repeat, zsend.complete, bytes, buflen);
+		}
+	} else {
+		zrecv.failure_total++;
+		if (zconf.output_module && zconf.output_module->other_ip) {
+			zconf.output_module->other_ip(
+					ip_hdr->saddr, ip_hdr->daddr,
+					r->name, is_repeat,	zsend.complete, bytes, buflen);
+		}
+	}
+
+	if (zconf.output_module && zconf.output_module->update
+			&& !(zrecv.success_unique % zconf.output_module->update_interval)) {
+		zconf.output_module->update(&zconf, &zsend, &zrecv);
+	}
+}
+
+int recv_update_pcap_stats(void)
+{
+	if (!pc) {
+		return EXIT_FAILURE;
+	}
+	struct pcap_stat pcst;
+	if (pcap_stats(pc, &pcst)) {
+		log_error("recv", "unable to retrieve pcap statistics: %s",
+				pcap_geterr(pc));
+		return EXIT_FAILURE;
+	} else {
+		zrecv.pcap_recv = pcst.ps_recv;
+		zrecv.pcap_drop = pcst.ps_drop;
+		zrecv.pcap_ifdrop = pcst.ps_ifdrop;
+	}
+	return EXIT_SUCCESS;
+}
+
+int recv_run(pthread_mutex_t *recv_ready_mutex)
+{
+	log_debug("recv", "thread started");
+	num_src_ports = zconf.source_port_last - zconf.source_port_first + 1;
+	ip_seen = calloc(IP_SEEN_SIZE, sizeof(uint64_t));
+	if (!ip_seen) {
+		log_fatal("recv", "couldn't allocate address bitmap");
+	}
+	log_debug("recv", "using dev %s", zconf.iface);
+	char errbuf[PCAP_ERRBUF_SIZE];
+	pc = pcap_open_live(zconf.iface, zconf.probe_module->pcap_snaplen,
+					PCAP_PROMISC, PCAP_TIMEOUT, errbuf);
+	if (pc == NULL) {
+		log_fatal("recv", "couldn't open device %s: %s",
+						zconf.iface, errbuf);
+	}
+	struct bpf_program bpf;
+	if (pcap_compile(pc, &bpf, zconf.probe_module->pcap_filter, 1, 0) < 0) {
+		log_fatal("recv", "couldn't compile filter");
+	}
+	if (pcap_setfilter(pc, &bpf) < 0) {
+		log_fatal("recv", "couldn't install filter");
+	}
+	log_debug("recv", "receiver ready");
+	pthread_mutex_lock(recv_ready_mutex);
+	zconf.recv_ready = 1;
+	pthread_mutex_unlock(recv_ready_mutex);
+	zrecv.start = now();
+	if (zconf.max_results == 0) {
+		zconf.max_results = -1;
+	}
+	do {
+		if (pcap_dispatch(pc, 0, packet_cb, NULL) == -1) {
+			log_fatal("recv", "pcap_dispatch error");
+		}
+		if (zconf.max_results && zrecv.success_unique >= zconf.max_results) {
+			zsend.complete = 1;
+			break;
+		}
+	} while (!(zsend.complete && (now()-zsend.finish > zconf.cooldown_secs)));
+	zrecv.finish = now();
+	// get final pcap statistics before closing
+	recv_update_pcap_stats();
+	pcap_close(pc);
+	zrecv.complete = 1;
+	log_debug("recv", "thread finished");
+	return 0;
+}
+

src/recv.h

@@ -0,0 +1,16 @@
+/*
+ * ZMap Copyright 2013 Regents of the University of Michigan 
+ * 
+ * Licensed under the Apache License, Version 2.0 (the "License"); you may not
+ * use this file except in compliance with the License. You may obtain a copy
+ * of the License at http://www.apache.org/licenses/LICENSE-2.0
+ */
+
+#ifndef _RECV_H
+#define _RECV_H
+
+int recv_update_pcap_stats(void);
+
+int recv_run();
+
+#endif //_RECV_H

src/send.c

@@ -0,0 +1,295 @@
+/*
+ * ZMap Copyright 2013 Regents of the University of Michigan 
+ * 
+ * Licensed under the Apache License, Version 2.0 (the "License"); you may not
+ * use this file except in compliance with the License. You may obtain a copy
+ * of the License at http://www.apache.org/licenses/LICENSE-2.0
+ */
+
+#include "send.h"
+
+#include <stdlib.h>
+#include <stdio.h>
+#include <stdint.h>
+#include <time.h>
+#include <unistd.h>
+#include <pthread.h>
+#include <string.h>
+#include <errno.h>
+#include <assert.h>
+
+#include <arpa/inet.h>
+#include <sys/socket.h>
+#include <sys/ioctl.h>
+#include <net/if.h>
+#include <linux/if_packet.h>
+
+#include "../lib/logger.h"
+#include "../lib/random.h"
+#include "../lib/blacklist.h"
+
+#include "cyclic.h"
+#include "state.h"
+#include "probe_modules/packet.h"
+#include "probe_modules/probe_modules.h"
+#include "validate.h"
+
+
+// lock to manage access to share send state (e.g. counters and cyclic)
+pthread_mutex_t send_mutex = PTHREAD_MUTEX_INITIALIZER;
+// lock to provide thread safety to the user provided send callback
+pthread_mutex_t syncb_mutex = PTHREAD_MUTEX_INITIALIZER;
+
+// globals to handle sending from multiple ip addresses (shared across threads)
+static uint16_t num_src_ports;
+static uint32_t num_addrs;
+static in_addr_t srcip_first;
+static in_addr_t srcip_last;
+// offset send addresses according to a random chosen per scan execution
+// in order to help prevent cross-scan interference
+static uint32_t srcip_offset;
+
+// global sender initialize (not thread specific)
+int send_init(void)
+{
+	// generate a new primitive root and starting position
+	cyclic_init(0, 0);
+	zsend.first_scanned = cyclic_get_curr_ip();
+
+	// compute number of targets
+	uint64_t allowed = blacklist_count_allowed();
+	if (allowed == (1LL << 32)) {
+		zsend.targets = 0xFFFFFFFF;
+	} else {
+		zsend.targets = allowed;
+	}
+	if (zsend.targets > zconf.max_targets) {
+		zsend.targets = zconf.max_targets;
+	}
+
+	// process the dotted-notation addresses passed to ZMAP and determine
+	// the source addresses from which we'll send packets;
+	srcip_first = inet_addr(zconf.source_ip_first);
+	if (srcip_first == INADDR_NONE) {
+		log_fatal("send", "invalid begin source ip address: `%s'",
+				zconf.source_ip_first);
+	}
+	srcip_last = inet_addr(zconf.source_ip_last);
+	if (srcip_last == INADDR_NONE) {
+		log_fatal("send", "invalid end source ip address: `%s'",
+				zconf.source_ip_last);
+	}
+	if (srcip_first == srcip_last) {
+		srcip_offset = 0;
+		num_addrs = 1;
+	} else {
+		srcip_offset = rand() % (srcip_last - srcip_first);
+		num_addrs = ntohl(srcip_last) - ntohl(srcip_first) + 1;
+	}
+
+	// process the source port range that ZMap is allowed to use
+	num_src_ports = zconf.source_port_last - zconf.source_port_first + 1;
+	log_debug("send", "will send from %i address%s on %u source ports",
+		 num_addrs, ((num_addrs==1)?"":"es"), num_src_ports);
+
+	// global initialization for send module
+	assert(zconf.probe_module);
+	if (zconf.probe_module->global_initialize) {
+		zconf.probe_module->global_initialize(&zconf);
+	}
+
+	// concert specified bandwidth to packet rate
+	if (zconf.bandwidth > 0) {
+		int pkt_len = zconf.probe_module->packet_length;
+		pkt_len *= 8;	
+		pkt_len += 8*24;	// 7 byte MAC preamble, 1 byte Start frame, 
+		                        // 4 byte CRC, 12 byte inter-frame gap
+		if (pkt_len < 84*8) {
+			pkt_len = 84*8;
+		}
+		if (zconf.bandwidth / pkt_len > 0xFFFFFFFF) {
+			zconf.rate = 0;
+		} else {
+			zconf.rate = zconf.bandwidth / pkt_len;
+			if (zconf.rate == 0) {
+				log_warn("send", "bandwidth %lu bit/s is slower than 1 pkt/s, "
+								"setting rate to 1 pkt/s", zconf.bandwidth);
+				zconf.rate = 1;
+			}
+		}
+		log_debug("send", "using bandwidth %lu bits/s, rate set to %d pkt/s",
+						zconf.bandwidth, zconf.rate);
+	}
+
+	if (zconf.dryrun) {
+		log_info("send", "dryrun mode -- won't actually send packets");
+	}
+
+	// initialize random validation key
+	validate_init();
+
+	zsend.start = now();	
+	return EXIT_SUCCESS;
+}
+
+static int get_socket(void)
+{
+	int sock = socket(AF_PACKET, SOCK_RAW, htons(ETH_P_ALL));
+	if (sock <= 0) {
+		log_fatal("send", "couldn't create socket. "
+			  "Are you root? Error: %s\n", strerror(errno));
+	}
+	return sock;
+}
+
+static inline ipaddr_n_t get_src_ip(ipaddr_n_t dst, int local_offset)
+{
+	if (srcip_first == srcip_last) {
+		return srcip_first;
+	}
+	return htonl(((ntohl(dst) + srcip_offset + local_offset) 
+			% num_addrs)) + srcip_first;
+}
+
+// one sender thread
+int send_run(void)
+{
+	log_debug("send", "thread started");
+	pthread_mutex_lock(&send_mutex);
+	int sock = get_socket();
+	struct sockaddr_ll sockaddr;
+	// get source interface index
+	struct ifreq if_idx;
+	memset(&if_idx, 0, sizeof(struct ifreq));
+	if (strlen(zconf.iface) >= IFNAMSIZ) {
+		log_error("send", "device interface name (%s) too long\n",
+				zconf.iface);
+		return -1;
+	}
+	strncpy(if_idx.ifr_name, zconf.iface, IFNAMSIZ-1);
+	if (ioctl(sock, SIOCGIFINDEX, &if_idx) < 0) {
+		perror("SIOCGIFINDEX");
+		return -1;
+	}
+	int ifindex = if_idx.ifr_ifindex;
+	// get source interface mac
+	struct ifreq if_mac;
+	memset(&if_mac, 0, sizeof(struct ifreq));
+	strncpy(if_mac.ifr_name, zconf.iface, IFNAMSIZ-1);
+	if (ioctl(sock, SIOCGIFHWADDR, &if_mac) < 0) {
+		perror("SIOCGIFHWADDR");
+		return -1;
+	}
+	// find source IP address associated with the dev from which we're sending.
+	// while we won't use this address for sending packets, we need the address
+	// to set certain socket options and it's easiest to just use the primary
+	// address the OS believes is associated.
+	struct ifreq if_ip;
+	memset(&if_ip, 0, sizeof(struct ifreq));
+	strncpy(if_ip.ifr_name, zconf.iface, IFNAMSIZ-1);
+	if (ioctl(sock, SIOCGIFADDR, &if_ip) < 0) {
+		perror("SIOCGIFADDR");
+		return -1;
+	}
+	// destination address for the socket
+	memset((void*) &sockaddr, 0, sizeof(struct sockaddr_ll));
+	sockaddr.sll_ifindex = ifindex;
+	sockaddr.sll_halen = ETH_ALEN;
+	memcpy(sockaddr.sll_addr, zconf.gw_mac, ETH_ALEN);
+
+	char buf[MAX_PACKET_SIZE];
+	memset(buf, 0, MAX_PACKET_SIZE);
+	zconf.probe_module->thread_initialize(buf, 
+					(unsigned char *)if_mac.ifr_hwaddr.sa_data, 
+					zconf.gw_mac, zconf.target_port);	
+	pthread_mutex_unlock(&send_mutex);
+
+	// adaptive timing to hit target rate
+	uint32_t count = 0;
+	uint32_t last_count = count;
+	double last_time = now();
+	uint32_t delay = 0;
+	int interval = 0;
+	volatile int vi;
+	if (zconf.rate > 0) {
+		// estimate initial rate
+		delay = 10000;
+		for (vi = delay; vi--; )
+			;
+		delay *= 1 / (now() - last_time) / (zconf.rate / zconf.senders);
+		interval = (zconf.rate / zconf.senders) / 20;
+		last_time = now();
+	}
+	while (1) {
+		// adaptive timing delay
+		if (delay > 0) {
+			count++;
+			for (vi = delay; vi--; )
+				;
+			if (!interval || (count % interval == 0)) {
+				double t = now();
+				delay *= (double)(count - last_count) 
+					/ (t - last_time) / (zconf.rate / zconf.senders);
+				if (delay < 1)
+					delay = 1;
+				last_count = count;
+				last_time = t;
+			}
+		}
+		// generate next ip from cyclic group and update global state
+		// (everything locked happens here)
+		pthread_mutex_lock(&send_mutex);
+		if (zsend.complete) {
+			pthread_mutex_unlock(&send_mutex);
+			break;
+		}
+		if (zsend.sent >= zconf.max_targets) {
+			zsend.complete = 1;
+			zsend.finish = now();
+			pthread_mutex_unlock(&send_mutex);
+			break;
+		}
+		if (zconf.max_runtime && zconf.max_runtime <= now() - zsend.start) {
+			zsend.complete = 1;
+			zsend.finish = now();
+			pthread_mutex_unlock(&send_mutex);
+			break;
+		}
+		uint32_t curr = cyclic_get_next_ip();
+		if (curr == zsend.first_scanned) {
+			zsend.complete = 1;
+			zsend.finish = now();
+		}
+		zsend.sent++;
+		pthread_mutex_unlock(&send_mutex);
+		for (int i=0; i < zconf.packet_streams; i++) {
+			uint32_t src_ip = get_src_ip(curr, i);
+
+		  	uint32_t validation[VALIDATE_BYTES/sizeof(uint32_t)];
+			validate_gen(src_ip, curr, (uint8_t *)validation);
+			zconf.probe_module->make_packet(buf, src_ip, curr, validation, i);
+
+			if (zconf.dryrun) {
+				zconf.probe_module->print_packet(stdout, buf);
+			} else {
+					int l = zconf.probe_module->packet_length;
+					int rc = sendto(sock, buf, 
+							l, 0,
+							(struct sockaddr *)&sockaddr,
+							sizeof(struct sockaddr_ll));
+					if (rc < 0) {
+						struct in_addr addr;
+						addr.s_addr = curr;
+						log_debug("send", "sendto failed for %s. %s",
+								  inet_ntoa(addr), strerror(errno));
+						pthread_mutex_lock(&send_mutex);
+						zsend.sendto_failures++;
+						pthread_mutex_unlock(&send_mutex);
+					}
+			}
+		}
+	}
+	log_debug("send", "thread finished");
+	return EXIT_SUCCESS;
+}
+

src/send.h

@@ -0,0 +1,15 @@
+/*
+ * ZMap Copyright 2013 Regents of the University of Michigan 
+ * 
+ * Licensed under the Apache License, Version 2.0 (the "License"); you may not
+ * use this file except in compliance with the License. You may obtain a copy
+ * of the License at http://www.apache.org/licenses/LICENSE-2.0
+ */
+
+#ifndef _SEND_H
+#define _SEND_H
+
+int send_init(void);
+int send_run(void);
+
+#endif //_SEND_H

src/state.c

@@ -0,0 +1,70 @@
+/*
+ * ZMap Copyright 2013 Regents of the University of Michigan 
+ * 
+ * Licensed under the Apache License, Version 2.0 (the "License"); you may not
+ * use this file except in compliance with the License. You may obtain a copy
+ * of the License at http://www.apache.org/licenses/LICENSE-2.0
+ */
+
+
+#include "state.h"
+#include "../lib/logger.h"
+
+// global configuration and defaults
+struct state_conf zconf = {
+	.log_level = LOG_INFO,
+	.source_port_first = 32768, // (these are the default
+	.source_port_last = 61000,  //   ephemeral range on Linux)
+	.output_filename = NULL,
+	.blacklist_filename = NULL,
+	.whitelist_filename = NULL,
+	.target_port = 0,
+	.max_targets = 0xFFFFFFFF,
+	.max_runtime = 0,
+	.max_results = 0,
+	.iface = NULL,
+	.rate = 0,
+	.bandwidth = 0,
+	.cooldown_secs = 0,
+	.senders = 1,
+	.packet_streams = 1,
+	.use_seed = 0,
+	.seed = 0,
+	.output_module = NULL,
+	.output_args = NULL,
+	.probe_module = NULL,
+	.probe_args = NULL,
+	.gw_mac = {0},
+	.gw_mac_set = 0,
+	.source_ip_first = NULL,
+	.source_ip_last = NULL,
+	.dryrun = 0,
+	.quiet = 0,
+	.summary = 0,
+	.recv_ready = 0,
+};
+
+// global sender stats and defaults
+struct state_send zsend = {
+	.start = 0.0,
+	.finish = 0.0,
+	.sent = 0,
+	.blacklisted = 0,
+	.complete = 0,
+	.sendto_failures = 0,
+	.targets = 0,
+};
+
+// global receiver stats and defaults
+struct state_recv zrecv = {
+	.success_unique = 0,
+	.success_total = 0,
+	.cooldown_unique = 0,
+	.cooldown_total = 0,
+	.failure_total = 0,
+	.complete = 0,
+	.pcap_recv = 0,
+	.pcap_drop = 0,
+	.pcap_ifdrop = 0,
+};
+

src/state.h

@@ -0,0 +1,124 @@
+/*
+ * ZMap Copyright 2013 Regents of the University of Michigan 
+ * 
+ * Licensed under the Apache License, Version 2.0 (the "License"); you may not
+ * use this file except in compliance with the License. You may obtain a copy
+ * of the License at http://www.apache.org/licenses/LICENSE-2.0
+ */
+
+#include <stdio.h>
+#include <stdint.h>
+#include <netinet/ip.h>
+#include <netinet/tcp.h>
+#include <netinet/ether.h>
+#include <net/if.h>
+
+#ifndef _STATE_H
+#define _STATE_H
+
+#define MAX_PACKET_SIZE 4096
+
+typedef uint32_t ipaddr_n_t; // IPv4 address network order
+typedef uint32_t ipaddr_h_t; // IPv4 address host order
+typedef uint16_t port_n_t; // port network order
+typedef uint16_t port_h_t; // port host order
+typedef unsigned char macaddr_t;
+
+struct probe_module;
+struct output_module;
+
+// global configuration
+struct state_conf {
+	int log_level;
+	port_h_t target_port;
+	port_h_t source_port_first;
+	port_h_t source_port_last;
+	// maximum number of packets that the scanner will send before terminating
+	uint32_t max_targets;
+	// maximum number of seconds that scanner will run before terminating
+	uint32_t max_runtime;
+	// maximum number of results before terminating
+	uint32_t max_results;
+	// name of network interface that
+   	// will be utilized for sending/receiving
+	char *iface;
+	// rate in packets per second
+	// that the sender will maintain
+	int rate;
+	// rate in bits per second
+	uint64_t bandwidth;
+	// how many seconds after the termination of the sender will the receiver
+	// continue to process responses
+	int cooldown_secs;
+	// number of sending threads
+	int senders;
+	// should use CLI provided randomization seed instead of generating
+	// a random seed.
+	int use_seed;
+	uint32_t seed;
+	// generator of the cyclic multiplicative group that is utilized for
+	// address generation
+	uint32_t generator;
+	int packet_streams;
+	struct probe_module *probe_module;
+	struct output_module *output_module;
+	char *probe_args;
+	char *output_args;
+	macaddr_t gw_mac[IFHWADDRLEN];
+  	int gw_mac_set;
+	char *source_ip_first;
+	char *source_ip_last;
+	char *output_filename;
+	char *blacklist_filename;
+	char *whitelist_filename;
+	int dryrun;
+	int summary;
+	int quiet;
+	int recv_ready;
+};
+extern struct state_conf zconf;
+
+// global sender stats
+struct state_send {
+	double start;
+	double finish;
+	uint32_t sent;
+	uint32_t blacklisted;
+	int complete;
+	uint32_t first_scanned;
+	uint32_t targets;
+	uint32_t sendto_failures;
+};
+extern struct state_send zsend;
+
+// global receiver stats
+struct state_recv {
+	// valid responses classified as "success"
+	uint32_t success_total;   
+	// unique IPs that sent valid responses classified as "success"
+	uint32_t success_unique;  
+	// valid responses classified as "success" received during cooldown
+	uint32_t cooldown_total;  
+	// unique IPs that first sent valid "success"es during cooldown
+	uint32_t cooldown_unique;
+	// valid responses NOT classified as "success"
+	uint32_t failure_total;   
+
+	int complete;  // has the scanner finished sending?	
+	double start;  // timestamp of when recv started	
+	double finish; // timestamp of when recv terminated
+
+	// number of packets captured by pcap filter
+	uint32_t pcap_recv;
+	// number of packets dropped because there was no room in
+	// the operating system's buffer when they arrived, because
+	// packets weren't being read fast enough
+	uint32_t pcap_drop;
+	// number of packets dropped by the network interface or its driver.
+	uint32_t pcap_ifdrop;
+
+};
+extern struct state_recv zrecv;
+
+#endif // _STATE_H
+

src/validate.c

@@ -0,0 +1,47 @@
+/*
+ * ZMap Copyright 2013 Regents of the University of Michigan 
+ * 
+ * Licensed under the Apache License, Version 2.0 (the "License"); you may not
+ * use this file except in compliance with the License. You may obtain a copy
+ * of the License at http://www.apache.org/licenses/LICENSE-2.0
+ */
+
+#include <stdint.h>
+#include <assert.h>
+#include "../lib/rijndael-alg-fst.h"
+#include "../lib/random.h"
+#include "../lib/logger.h"
+#include "validate.h"
+
+#define AES_ROUNDS 10
+#define AES_BLOCK_WORDS	4
+#define AES_KEY_BYTES 16
+
+static int inited = 0;
+static uint32_t aes_input[AES_BLOCK_WORDS];
+static uint32_t aes_sched[(AES_ROUNDS+1)*4];
+
+void validate_init()
+{
+	for (int i=0; i < AES_BLOCK_WORDS; i++) {
+		aes_input[i] = 0;
+	}
+	uint8_t key[AES_KEY_BYTES];
+	if (!random_bytes(key, AES_KEY_BYTES)) {
+		log_fatal("validate", "couldn't get random bytes");
+	}
+	if (rijndaelKeySetupEnc(aes_sched, key, AES_KEY_BYTES*8) != AES_ROUNDS) {
+		log_fatal("validate", "couldn't initialize AES key");
+	}
+	inited = 1;
+}
+
+void validate_gen(const uint32_t src, const uint32_t dst,
+				uint8_t output[VALIDATE_BYTES])
+{
+	assert(inited);
+	aes_input[0] = src;
+	aes_input[1] = dst;
+	rijndaelEncrypt(aes_sched, AES_ROUNDS, (uint8_t *)aes_input, output);
+}
+

src/validate.h

@@ -0,0 +1,17 @@
+/*
+ * ZMap Copyright 2013 Regents of the University of Michigan 
+ * 
+ * Licensed under the Apache License, Version 2.0 (the "License"); you may not
+ * use this file except in compliance with the License. You may obtain a copy
+ * of the License at http://www.apache.org/licenses/LICENSE-2.0
+ */
+
+#ifndef _VALIDATE_H
+#define _VALIDATE_H
+
+#define VALIDATE_BYTES 16
+
+void validate_init();
+void validate_gen(const uint32_t src, const uint32_t dst, uint8_t output[VALIDATE_BYTES]);
+
+#endif//_VALIDATE_H

src/zmap.1

@@ -0,0 +1,155 @@
+.TH zmap 1 "06 Aug 2013" "1.0" "zmap man page"
+.SH NAME
+zmap \- A fast Internet-wide scanner
+.SH SYNOPSIS
+.B zmap 
+[ 
+.I "OPTIONS \&..."
+] 
+.SH DESCRIPTION
+.I ZMap 
+is a network tool for scanning the entire Internet (or large samples).
+.SH OPTIONS
+.SS "Basic options"
+.TP
+.B  \-p, --target-port=port
+TCP port number to scan (for SYN scans)
+.TP
+.B \-o, --output-file=name
+When using an output module that uses a file (such as the default),
+write results to this file. Use - for stdout.
+.TP
+.B \-b, --blacklist-file=path
+File of subnets to exclude, in CIDR notation (e.g. 192.168.0.0/16), 
+one-per line. It is recommended you use this to exclude RFC 1918
+addresses, multicast, IANA reserved space, and other IANA
+special-purpose addresses. An example blacklist file is provided in
+.B conf/blacklist.example
+for this purpose.
+.TP
+.B  -w, --whitelist-file=path
+File of subnets to constrain scan to, in CIDR 
+notation, e.g. 192.168.0.0/16
+
+.SS "Scan options"
+.TP
+.B \-n, --max-targets=n
+Cap number of targets to probe (as a number or 
+a percentage of the address space)
+.TP
+.B \-N, --max-results=n
+Cap number of results to return
+.TP
+.B \-t, --max-runtime=secs
+Cap length of time for sending packets
+.TP
+.B \-r, --rate=pps
+Set send rate in packets/sec
+.TP
+.B \-B, --bandwidth=bps
+Set send rate in bits/second (supports suffixes G, M and K). This 
+overrides the
+.B --rate
+flag.
+.TP
+.B \-c, --cooldown-time=secs
+How long to continue receiving after sending 
+last probe  (default=8)
+.TP
+.B \-e, --seed=n
+Seed used to select address permutation.  Specify the same seed in order to scan the same sample repeatedly.
+.TP
+.B \-T, --sender-threads=n
+Threads used to send packets  (default=1)
+.TP
+.B \-P, --probes=n
+Number of probes to send to each IP  
+(default=1)
+.TP
+.B \-d, --dryrun
+Print out each packet to stdout instead of sending it.
+(May be useful for debugging.)
+
+.SS "Network options"
+.TP
+.B \-s, --source-port=port|range
+Source port(s) for scan packets
+.TP
+.B \-S, --source-ip=ip|range
+Source address(es) for scan packets
+.TP
+.B \-G, --gateway-mac=addr
+Specify gateway MAC address. All packets will be sent to this
+Ethernet address.
+.TP
+.B \-i, --interface=name
+Specify network interface to use.
+
+.SS "Advanced options"
+.TP
+.B \-M, --probe-module=name
+Select probe module  (default=tcp_synscan)
+.TP
+.B \-O, --output-module=name
+Select output module  (default=simple_file)
+.TP
+.B --probe-args=args
+Arguments to pass to probe module
+.TP
+.B --output-args=args
+Arguments to pass to output module
+.TP
+.B --list-output-modules
+List available output modules
+.TP
+.B --list-probe-modules
+List available probe modules
+
+.SS "Additional options"
+.TP
+.B \-C, --config=filename
+Read a configuration file, which can specify 
+any of these options  (default=zmap.conf)
+.TP
+.B \-q, --quiet
+Do not print status updates
+.TP
+.B \-g, --summary
+Print configuration and summary at end of scan
+.TP
+.B \-v, --verbosity=n
+Level of log detail (0-5)  (default=3)
+.TP
+.B \-h, --help
+Print help and exit
+.TP
+.B \-V, --version
+Print version and exit
+
+.SH EXAMPLES
+Scan the whole Internet for hosts with port 443 open (results discarded):
+.PP
+.\" -p: example of
+.B zmap \-p 443
+.PP
+Find 5 HTTP servers (port 80), scanning at 10 Mb/s, print the results to stdout:
+.PP
+.\" -N: example of
+.\" -B: example of
+.B zmap -N 5 -B 10M -p 80 -o -
+
+.SH WARNING
+By default, ZMap attempts to scan at the line speed of your Ethernet
+interface and can easily use 1 Gbit/second of bandwidth. If your
+network is not able to support sending packets this quickly, your
+local network may become congested, causing connectivity problems for
+you and those around you.  Use the -B (--bandwidth) option to set
+ZMap's maximum bandwidth to an appropriate limit for your network and
+upstream connection.
+.
+.SH AUTHOR
+Zakir Durumeric,
+Eric Wustrow,
+J. Alex Halderman
+.B (https://www.zmap.io)
+

src/zmap.c

@@ -0,0 +1,483 @@
+/*
+ * ZMap Copyright 2013 Regents of the University of Michigan 
+ * 
+ * Licensed under the Apache License, Version 2.0 (the "License"); you may not
+ * use this file except in compliance with the License. You may obtain a copy
+ * of the License at http://www.apache.org/licenses/LICENSE-2.0
+ */
+
+#define _GNU_SOURCE
+
+#include <stdlib.h>
+#include <stdio.h>
+
+#include <assert.h>
+#include <sched.h>
+#include <unistd.h>
+#include <errno.h>
+#include <string.h>
+#include <net/if.h>
+#include <sys/socket.h>
+#include <netinet/in.h>
+#include <arpa/inet.h>
+
+#include <pcap/pcap.h>
+#include <pthread.h>
+
+#include "../lib/logger.h"
+#include "../lib/random.h"
+
+#include "zopt.h"
+#include "send.h"
+#include "recv.h"
+#include "state.h"
+#include "monitor.h"
+#include "output_modules/output_modules.h"
+#include "probe_modules/probe_modules.h"
+#include "get_gateway.h"
+
+pthread_mutex_t cpu_affinity_mutex = PTHREAD_MUTEX_INITIALIZER;
+pthread_mutex_t recv_ready_mutex = PTHREAD_MUTEX_INITIALIZER;
+
+static void set_cpu(void)
+{
+	pthread_mutex_lock(&cpu_affinity_mutex);
+	static int core=0;
+	int num_cores = sysconf(_SC_NPROCESSORS_ONLN);
+	cpu_set_t cpuset;	
+	CPU_ZERO(&cpuset);
+	CPU_SET(core, &cpuset);
+	if (pthread_setaffinity_np(pthread_self(),
+				sizeof(cpu_set_t), &cpuset) != 0) {
+		log_error("zmap", "can't set thread CPU affinity");
+	}
+	log_trace("zmap", "set thread %u affinity to core %d",
+			pthread_self(), core);
+	core = (core + 1) % num_cores;
+	pthread_mutex_unlock(&cpu_affinity_mutex);
+}	
+
+static void* start_send(__attribute__((unused)) void *arg)
+{
+	set_cpu();
+	send_run();
+	return NULL;
+}
+
+static void* start_recv(__attribute__((unused)) void *arg)
+{
+	set_cpu();
+	recv_run(&recv_ready_mutex);
+	return NULL;
+}
+
+static void *start_mon(__attribute__((unused)) void *arg)
+{
+	set_cpu();
+	monitor_run();
+	return NULL;
+}
+
+#define SI(w,x,y) printf("%s\t%s\t%i\n", w, x, y);
+#define SD(w,x,y) printf("%s\t%s\t%f\n", w, x, y);
+#define SU(w,x,y) printf("%s\t%s\t%u\n", w, x, y);
+#define SLU(w,x,y) printf("%s\t%s\t%lu\n", w, x, y);
+#define SS(w,x,y) printf("%s\t%s\t%s\n", w, x, y);
+#define STRTIME_LEN 1024
+
+static void summary(void)
+{
+	char send_start_time[STRTIME_LEN+1];
+	assert(dstrftime(send_start_time, STRTIME_LEN, "%c", zsend.start));
+	char send_end_time[STRTIME_LEN+1];
+	assert(dstrftime(send_end_time, STRTIME_LEN, "%c", zsend.finish));
+	char recv_start_time[STRTIME_LEN+1];
+	assert(dstrftime(recv_start_time, STRTIME_LEN, "%c", zrecv.start));
+	char recv_end_time[STRTIME_LEN+1];
+	assert(dstrftime(recv_end_time, STRTIME_LEN, "%c", zrecv.finish));
+	double hitrate = ((double) 100 * zrecv.success_unique)/((double)zsend.sent);
+
+	SU("cnf", "target-port", zconf.target_port);
+	SU("cnf", "source-port-range-begin", zconf.source_port_first);
+	SU("cnf", "source-port-range-end", zconf.source_port_last);
+	SS("cnf", "source-addr-range-begin", zconf.source_ip_first);
+	SS("cnf", "source-addr-range-end", zconf.source_ip_last);
+	SU("cnf", "maximum-targets", zconf.max_targets);
+	SU("cnf", "maximum-runtime", zconf.max_runtime);
+	SU("cnf", "maximum-results", zconf.max_results);
+	SU("cnf", "permutation-seed", zconf.seed);
+	SI("cnf", "cooldown-period", zconf.cooldown_secs);
+	SS("cnf", "send-interface", zconf.iface);
+	SI("cnf", "rate", zconf.rate);
+	SLU("cnf", "bandwidth", zconf.bandwidth);
+	SU("env", "nprocessors", (unsigned) sysconf(_SC_NPROCESSORS_ONLN));
+	SS("exc", "send-start-time", send_start_time);
+	SS("exc", "send-end-time", send_end_time);
+	SS("exc", "recv-start-time", recv_start_time);
+	SS("exc", "recv-end-time", recv_end_time);
+	SU("exc", "sent", zsend.sent);
+	SU("exc", "blacklisted", zsend.blacklisted);
+	SU("exc", "first-scanned", zsend.first_scanned);
+	SD("exc", "hit-rate", hitrate);
+	SU("exc", "success-total", zrecv.success_total);
+	SU("exc", "success-unique", zrecv.success_unique);
+	SU("exc", "success-cooldown-total", zrecv.cooldown_total);
+	SU("exc", "success-cooldown-unique", zrecv.cooldown_unique);
+	SU("exc", "failure-total", zrecv.failure_total);
+	SU("exc", "sendto-failures", zsend.sendto_failures);
+	SU("adv", "permutation-gen", zconf.generator);
+	SS("exc", "scan-type", zconf.probe_module->name);
+}
+
+static void start_zmap(void)
+{
+	log_init(stderr, zconf.log_level);
+	log_info("zmap", "started");
+
+	// finish setting up configuration
+	if (zconf.iface == NULL) {
+		char errbuf[PCAP_ERRBUF_SIZE];
+		char *iface = pcap_lookupdev(errbuf);
+		if (iface == NULL) {
+			log_fatal("zmap", "could not detect default network interface "
+					"(e.g. eth0). Try running as root or setting"
+					" interface using -i flag.");
+		}
+		log_debug("zmap", "no interface provided. will use %s", iface);
+		zconf.iface = iface;
+	}
+	if (zconf.source_ip_first == NULL) {
+		struct in_addr default_ip;
+		zconf.source_ip_first = malloc(INET_ADDRSTRLEN);
+		zconf.source_ip_last = zconf.source_ip_first;
+		if (get_iface_ip(zconf.iface, &default_ip) < 0) {
+			log_fatal("zmap", "could not detect default IP address for for %s."
+					" Try specifying a source address (-S).", zconf.iface);
+		}
+		inet_ntop(AF_INET, &default_ip, zconf.source_ip_first, INET_ADDRSTRLEN);
+		log_debug("zmap", "no source IP address given. will use %s",
+				zconf.source_ip_first);
+	}
+	if (!zconf.gw_mac_set) {
+		struct in_addr gw_ip;
+		char iface[IF_NAMESIZE];
+		if (get_default_gw(&gw_ip, iface) < 0) {
+			log_fatal("zmap", "could not detect default gateway address for %i."
+					" Try setting default gateway mac address (-G).");
+		}
+		log_debug("zmap", "found gateway IP %s on %s", inet_ntoa(gw_ip), iface); 
+		if (get_hw_addr(&gw_ip, iface, zconf.gw_mac) < 0) {
+			log_fatal("zmap", "could not detect GW MAC address for %s on %s."
+					" Try setting default gateway mac address (-G).",
+					inet_ntoa(gw_ip), zconf.iface);
+		}
+		zconf.gw_mac_set = 1;
+		log_debug("zmap", "using default gateway MAC %02x:%02x:%02x:%02x:%02x:%02x",
+				  zconf.gw_mac[0], zconf.gw_mac[1], zconf.gw_mac[2],
+				  zconf.gw_mac[3], zconf.gw_mac[4], zconf.gw_mac[5]);
+	}
+
+	// initialization
+	if (zconf.output_module && zconf.output_module->init) {
+		zconf.output_module->init(&zconf);
+	}
+	if (send_init()) {
+		exit(EXIT_FAILURE);
+	}
+	if (zconf.output_module && zconf.output_module->start) {
+		zconf.output_module->start(&zconf, &zsend, &zrecv);
+	}
+
+	// start threads
+	pthread_t *tsend, trecv, tmon;
+	int r = pthread_create(&trecv, NULL, start_recv, NULL);
+	if (r != 0) {
+		log_fatal("zmap", "unable to create recv thread");
+		exit(EXIT_FAILURE);
+	}
+	for (;;) {
+		pthread_mutex_lock(&recv_ready_mutex);
+		if (zconf.recv_ready) {
+			break;
+		}
+		pthread_mutex_unlock(&recv_ready_mutex);
+	}
+	tsend = malloc(zconf.senders * sizeof(pthread_t));
+	assert(tsend);
+	log_debug("zmap", "using %d sender threads", zconf.senders);
+	for (int i=0; i < zconf.senders; i++) {
+		r = pthread_create(&tsend[i], NULL, start_send, NULL);
+		if (r != 0) {
+			log_fatal("zmap", "unable to create send thread");
+			exit(EXIT_FAILURE);
+		}
+	}
+	if (!zconf.quiet) {
+		r = pthread_create(&tmon, NULL, start_mon, NULL);
+		if (r != 0) {
+			log_fatal("zmap", "unable to create monitor thread");
+			exit(EXIT_FAILURE);
+		}
+	}
+
+	// wait for completion
+	for (int i=0; i < zconf.senders; i++) {
+		pthread_join(tsend[i], NULL);
+		if (r != 0) {
+			log_fatal("zmap", "unable to join send thread");
+			exit(EXIT_FAILURE);
+		}
+	}
+	log_debug("zmap", "senders finished");
+	pthread_join(trecv, NULL);
+	if (r != 0) {
+		log_fatal("zmap", "unable to join recv thread");
+		exit(EXIT_FAILURE);
+	}
+	if (!zconf.quiet) {
+		pthread_join(tmon, NULL);
+		if (r != 0) {
+			log_fatal("zmap", "unable to join monitor thread");
+			exit(EXIT_FAILURE);
+		}
+	}
+
+	// finished
+	if (zconf.summary) {
+		summary();
+	}
+	if (zconf.output_module && zconf.output_module->close) {
+		zconf.output_module->close(&zconf, &zsend, &zrecv);
+	}
+	if (zconf.probe_module && zconf.probe_module->close) {
+		zconf.probe_module->close(&zconf, &zsend, &zrecv);
+	}
+	log_info("zmap", "completed");
+}
+
+static void enforce_range(const char *name, int v, int min, int max)
+{
+	if (v < min || v > max) {
+	  	fprintf(stderr, "%s: argument `%s' must be between %d and %d\n",
+			CMDLINE_PARSER_PACKAGE, name, min, max);
+		exit(EXIT_FAILURE);
+	}
+}
+
+static int file_exists(char *name)
+{
+	FILE *file = fopen(name, "r");
+	if (!file)
+		return 0;
+	fclose(file);
+	return 1;
+}
+
+#define MAC_LEN IFHWADDRLEN
+int parse_mac(macaddr_t *out, char *in)
+{
+	if (strlen(in) < MAC_LEN*3-1)
+		return 0;
+	char octet[3];
+	octet[2] = '\0';
+	for (int i=0; i < MAC_LEN; i++) {
+		if (i < MAC_LEN-1 && in[i*3+2] != ':') {
+			return 0;
+		}
+		strncpy(octet, &in[i*3], 2);
+		char *err = NULL;
+		long b = strtol(octet, &err, 16);
+		if (err && *err != '\0') {
+			return 0;
+		}
+		out[i] = b & 0xFF;
+	}
+	return 1;
+}
+
+#define SET_IF_GIVEN(DST,ARG) \
+	{ if (args.ARG##_given) { (DST) = args.ARG##_arg; }; }
+#define SET_BOOL(DST,ARG) \
+	{ if (args.ARG##_given) { (DST) = 1; }; }
+
+int main(int argc, char *argv[])
+{
+	struct gengetopt_args_info args;
+	struct cmdline_parser_params *params;
+	params = cmdline_parser_params_create();
+	params->initialize = 1;
+	params->override = 0;
+	params->check_required = 0;
+	if (cmdline_parser_ext(argc, argv, &args, params) != 0) {
+		exit(EXIT_SUCCESS);
+	}
+	if (args.help_given) {
+		cmdline_parser_print_help();
+		exit(EXIT_SUCCESS);
+	}
+	if (args.version_given) {
+		cmdline_parser_print_version();
+		exit(EXIT_SUCCESS);
+	}
+	if (args.list_output_modules_given) {
+		print_output_modules();
+		exit(EXIT_SUCCESS);
+	}
+	if (args.list_probe_modules_given) {
+		print_probe_modules();
+		exit(EXIT_SUCCESS);
+	}
+	if (args.config_given || file_exists(args.config_arg)) {
+		params->initialize = 0;
+		params->override = 0;
+		if (cmdline_parser_config_file(args.config_arg, &args, params) 
+				!= 0) {
+			exit(EXIT_FAILURE);
+		}
+	}
+	if (cmdline_parser_required(&args, CMDLINE_PARSER_PACKAGE) != 0) {
+		exit(EXIT_FAILURE);
+	}
+
+	SET_IF_GIVEN(zconf.output_filename, output_file);
+	SET_IF_GIVEN(zconf.blacklist_filename, blacklist_file);
+	SET_IF_GIVEN(zconf.whitelist_filename, whitelist_file);
+	SET_IF_GIVEN(zconf.probe_args, probe_args);
+	SET_IF_GIVEN(zconf.output_args, output_args);
+	SET_IF_GIVEN(zconf.iface, interface);
+	SET_IF_GIVEN(zconf.max_runtime, max_runtime);
+	SET_IF_GIVEN(zconf.max_results, max_results);
+	SET_IF_GIVEN(zconf.rate, rate);
+	SET_IF_GIVEN(zconf.packet_streams, probes);
+	SET_BOOL(zconf.dryrun, dryrun);
+	SET_BOOL(zconf.quiet, quiet);
+	SET_BOOL(zconf.summary, summary);
+	zconf.cooldown_secs = args.cooldown_time_arg;
+	zconf.senders = args.sender_threads_arg;
+	zconf.log_level = args.verbosity_arg;
+
+	zconf.output_module = get_output_module_by_name(args.output_module_arg);
+	if (!zconf.output_module) {
+	  fprintf(stderr, "%s: specified output module (%s) does not exist\n",
+		  args.output_module_arg, CMDLINE_PARSER_PACKAGE);
+	  exit(EXIT_FAILURE);
+	}
+	zconf.probe_module = get_probe_module_by_name(args.probe_module_arg);
+	if (!zconf.probe_module) {
+		fprintf(stderr, "%s: specified probe module (%s) does not exist\n",
+				CMDLINE_PARSER_PACKAGE, args.probe_module_arg);
+	  exit(EXIT_FAILURE);
+	}
+	if (zconf.probe_module->port_args) {
+		if (args.source_port_given) {
+			char *dash = strchr(args.source_port_arg, '-');
+			if (dash) { // range
+				*dash = '\0';
+				zconf.source_port_first = atoi(args.source_port_arg);
+				enforce_range("starting source-port", zconf.source_port_first, 0, 0xFFFF);
+				zconf.source_port_last = atoi(dash+1);
+				enforce_range("ending source-port", zconf.source_port_last, 0, 0xFFFF);
+				if (zconf.source_port_first > zconf.source_port_last) {
+					fprintf(stderr, "%s: invalid source port range: "
+						"last port is less than first port\n",
+						CMDLINE_PARSER_PACKAGE);
+					exit(EXIT_FAILURE);
+				}
+			} else { // single port
+				int port = atoi(args.source_port_arg);
+				enforce_range("source-port", port, 0, 0xFFFF);
+				zconf.source_port_first = port;
+				zconf.source_port_last = port;
+			}
+		}
+	  	if (!args.target_port_given) {
+			fprintf(stderr, "%s: target port is required for this type of probe\n",
+				CMDLINE_PARSER_PACKAGE);
+			exit(EXIT_FAILURE);
+		}
+		enforce_range("target-port", args.target_port_arg, 0, 0xFFFF);
+		zconf.target_port = args.target_port_arg;
+	}
+	if (args.source_ip_given) {
+		char *dash = strchr(args.source_ip_arg, '-');
+		if (dash) { // range
+			*dash = '\0';
+			zconf.source_ip_first = args.source_ip_arg;
+			zconf.source_ip_last = dash+1;
+		} else { // single address
+			zconf.source_ip_first = args.source_ip_arg;
+			zconf.source_ip_last = args.source_ip_arg;
+		}
+	}
+	if (args.gateway_mac_given) {
+		if (!parse_mac(zconf.gw_mac, args.gateway_mac_arg)) {
+			fprintf(stderr, "%s: invalid MAC address `%s'\n", 
+				CMDLINE_PARSER_PACKAGE, args.gateway_mac_arg);
+			exit(EXIT_FAILURE);
+		}
+		zconf.gw_mac_set = 1;
+	}
+	if (args.seed_given) {
+		zconf.seed = args.seed_arg;
+		zconf.use_seed = 1;
+	}
+	if (args.bandwidth_given) {
+		// Supported: G,g=*1000000000; M,m=*1000000 K,k=*1000 bits per second
+		zconf.bandwidth = atoi(args.bandwidth_arg);
+		char *suffix = args.bandwidth_arg;
+		while (*suffix >= '0' && *suffix <= '9') {
+			suffix++;
+		}
+		if (*suffix) {
+			switch (*suffix) {
+			case 'G': case 'g':
+				zconf.bandwidth *= 1000000000;
+				break;
+			case 'M': case 'm':
+				zconf.bandwidth *= 1000000;
+				break;
+			case 'K': case 'k':
+				zconf.bandwidth *= 1000;
+				break;
+			default:
+			  	fprintf(stderr, "%s: unknown bandwidth suffix '%s' "
+					"(supported suffixes are G, M and K)\n",
+					CMDLINE_PARSER_PACKAGE, suffix);
+				exit(EXIT_FAILURE);
+			}
+		}
+	}
+	if (args.max_targets_given) {
+		errno = 0;
+		char *end;
+	  	double v = strtod(args.max_targets_arg, &end);
+		if (end == args.max_targets_arg || errno != 0) {
+			fprintf(stderr, "%s: can't convert max-targets to a number\n",
+					CMDLINE_PARSER_PACKAGE);
+			exit(EXIT_FAILURE);
+		}
+		if (end[0] == '%' && end[1] == '\0') {
+			// treat as percentage
+			v = v * (1L << 32) / 100.;
+		} else if (end[0] != '\0') {
+			fprintf(stderr, "%s: extra characters after max-targets\n",
+				  CMDLINE_PARSER_PACKAGE);
+			exit(EXIT_FAILURE);
+		}
+		if (v <= 0) {
+			zconf.max_targets = 0;
+		}
+		else if (v >= (1L << 32)) {
+			zconf.max_targets = 0xFFFFFFFF;
+		} else {
+			zconf.max_targets = v;
+		}
+	}
+
+	start_zmap();
+
+	cmdline_parser_free(&args);
+	free(params);
+	return EXIT_SUCCESS;
+}
+

src/zopt.ggo

@@ -0,0 +1,119 @@
+# ZMap Copyright 2013 Regents of the University of Michigan
+
+# Licensed under the Apache License, Version 2.0 (the "License"); you may not use this file except in compliance with the License. You may obtain a copy of the License at http://www.apache.org/licenses/LICENSE-2.0
+
+# zmap option description to be processed by gengetopt
+
+package "zmap"
+version "1.0.0"
+purpose "A fast Internet-wide scanner."
+
+section "Basic arguments"
+
+option "target-port"    	p "TCP port number to scan (for SYN scans)"
+	   typestr="port"
+	   optional int
+option "output-file"  		o "Output file"
+	   typestr="name"
+	   optional string
+option "blacklist-file"		b "File of subnets to exclude, in CIDR notation, e.g. 192.168.0.0/16"
+	   typestr="path"
+	   optional string
+option "whitelist-file"		w "File of subnets to constrain scan to, in CIDR notation, e.g. 192.168.0.0/16"
+	   typestr="path"
+	   optional string
+
+section "Scan options"
+
+option "max-targets"		n "Cap number of targets to probe (as a number or a percentage of the address space)"
+	   typestr="n"
+	   optional string
+option "max-results"		N "Cap number of results to return"
+	   typestr="n"
+	   optional int
+option "max-runtime"		t "Cap length of time for sending packets"
+	   typestr="ses"
+	   optional int
+option "rate"           	r "Set send rate in packets/sec"
+	   typestr="pps"
+	   optional int
+option "bandwidth"			B "Set send rate in bits/second (supports suffixes G, M and K)"
+	   typestr="bps"
+	   optional string
+option "cooldown-time"  	c "How long to continue receiving after sending last probe"
+	   typestr="secs"
+	   default="8"
+	   optional int
+option "seed"           	e "Seed used to select address permutation"
+	   typestr="n"
+	   optional int
+option "sender-threads"		T "Threads used to send packets"
+	   typestr="n"
+	   default="1"
+	   optional int
+option "probes"				P "Number of probes to send to each IP"
+	   typestr="n"
+	   default="1"
+	   optional int
+option "dryrun"				d "Don't actually send packets"
+	   optional
+
+section "Network options" 
+
+option "source-port"    	s "Source port(s) for scan packets"
+	   typestr="port|range"
+	   optional string
+option "source-ip"			S "Source address(es) for scan packets"
+	   typestr="ip|range"
+	   optional string
+option "gateway-mac"		G "Specify gateway MAC address"
+	   typestr="addr"
+	   optional string
+option "interface"			i "Specify network interface to use"
+	   typestr="name"
+	   optional string
+
+section "Advanced options"
+
+option "probe-module"		M "Select probe module"
+	   typestr="name"
+	   default="tcp_synscan"
+	   optional string
+option "output-module"		O "Select output module"
+	   typestr="name"
+	   default="simple_file"
+	   optional string
+option "probe-args"			- "Arguments to pass to probe module"
+	   typestr="args"
+	   optional string
+option "output-args"		- "Arguments to pass to output module"
+	   typestr="args"
+	   optional string
+option "list-output-modules" - "List available output modules"
+           optional
+option "list-probe-modules"	 - "List available probe modules"
+           optional
+
+section "Additional options"
+
+option "config"				C "Read a configuration file, which can specify any of these options"
+	   typestr="filename"
+	   default="/etc/zmap/zmap.conf"
+	   optional string
+option "quiet"				q "Do not print status updates"
+       optional
+option "summary"			g "Print configuration and summary at end of scan"
+       optional
+option "verbosity"			v "Level of log detail (0-5)"
+	   typestr="n"
+	   default="3"
+	   optional int
+option "help"				h "Print help and exit"
+	   optional
+option "version"			V "Print version and exit"
+	   optional
+
+text "\nExamples:\n\
+     zmap -p 443  (scans the whole Internet for hosts with port 443 open)\n\
+     zmap -N 5 -B 10M -p 80 -o -  (find 5 HTTP servers, scanning at 10 Mb/s)"
+

src/zopt.h

@@ -0,0 +1,298 @@
+/** @file zopt.h
+ *  @brief The header file for the command line option parser
+ *  generated by GNU Gengetopt version 2.22.5
+ *  http://www.gnu.org/software/gengetopt.
+ *  DO NOT modify this file, since it can be overwritten
+ *  @author GNU Gengetopt by Lorenzo Bettini */
+
+#ifndef ZOPT_H
+#define ZOPT_H
+
+/* If we use autoconf.  */
+#ifdef HAVE_CONFIG_H
+#include "config.h"
+#endif
+
+#include <stdio.h> /* for FILE */
+
+#ifdef __cplusplus
+extern "C" {
+#endif /* __cplusplus */
+
+#ifndef CMDLINE_PARSER_PACKAGE
+/** @brief the program name (used for printing errors) */
+#define CMDLINE_PARSER_PACKAGE "zmap"
+#endif
+
+#ifndef CMDLINE_PARSER_PACKAGE_NAME
+/** @brief the complete program name (used for help and version) */
+#define CMDLINE_PARSER_PACKAGE_NAME "zmap"
+#endif
+
+#ifndef CMDLINE_PARSER_VERSION
+/** @brief the program version */
+#define CMDLINE_PARSER_VERSION "1.0.0"
+#endif
+
+/** @brief Where the command line options are stored */
+struct gengetopt_args_info
+{
+  int target_port_arg;	/**< @brief TCP port number to scan (for SYN scans).  */
+  char * target_port_orig;	/**< @brief TCP port number to scan (for SYN scans) original value given at command line.  */
+  const char *target_port_help; /**< @brief TCP port number to scan (for SYN scans) help description.  */
+  char * output_file_arg;	/**< @brief Output file.  */
+  char * output_file_orig;	/**< @brief Output file original value given at command line.  */
+  const char *output_file_help; /**< @brief Output file help description.  */
+  char * blacklist_file_arg;	/**< @brief File of subnets to exclude, in CIDR notation, e.g. 192.168.0.0/16.  */
+  char * blacklist_file_orig;	/**< @brief File of subnets to exclude, in CIDR notation, e.g. 192.168.0.0/16 original value given at command line.  */
+  const char *blacklist_file_help; /**< @brief File of subnets to exclude, in CIDR notation, e.g. 192.168.0.0/16 help description.  */
+  char * whitelist_file_arg;	/**< @brief File of subnets to constrain scan to, in CIDR notation, e.g. 192.168.0.0/16.  */
+  char * whitelist_file_orig;	/**< @brief File of subnets to constrain scan to, in CIDR notation, e.g. 192.168.0.0/16 original value given at command line.  */
+  const char *whitelist_file_help; /**< @brief File of subnets to constrain scan to, in CIDR notation, e.g. 192.168.0.0/16 help description.  */
+  char * max_targets_arg;	/**< @brief Cap number of targets to probe (as a number or a percentage of the address space).  */
+  char * max_targets_orig;	/**< @brief Cap number of targets to probe (as a number or a percentage of the address space) original value given at command line.  */
+  const char *max_targets_help; /**< @brief Cap number of targets to probe (as a number or a percentage of the address space) help description.  */
+  int max_results_arg;	/**< @brief Cap number of results to return.  */
+  char * max_results_orig;	/**< @brief Cap number of results to return original value given at command line.  */
+  const char *max_results_help; /**< @brief Cap number of results to return help description.  */
+  int max_runtime_arg;	/**< @brief Cap length of time for sending packets.  */
+  char * max_runtime_orig;	/**< @brief Cap length of time for sending packets original value given at command line.  */
+  const char *max_runtime_help; /**< @brief Cap length of time for sending packets help description.  */
+  int rate_arg;	/**< @brief Set send rate in packets/sec.  */
+  char * rate_orig;	/**< @brief Set send rate in packets/sec original value given at command line.  */
+  const char *rate_help; /**< @brief Set send rate in packets/sec help description.  */
+  char * bandwidth_arg;	/**< @brief Set send rate in bits/second (supports suffixes G, M and K).  */
+  char * bandwidth_orig;	/**< @brief Set send rate in bits/second (supports suffixes G, M and K) original value given at command line.  */
+  const char *bandwidth_help; /**< @brief Set send rate in bits/second (supports suffixes G, M and K) help description.  */
+  int cooldown_time_arg;	/**< @brief How long to continue receiving after sending last probe (default='8').  */
+  char * cooldown_time_orig;	/**< @brief How long to continue receiving after sending last probe original value given at command line.  */
+  const char *cooldown_time_help; /**< @brief How long to continue receiving after sending last probe help description.  */
+  int seed_arg;	/**< @brief Seed used to select address permutation.  */
+  char * seed_orig;	/**< @brief Seed used to select address permutation original value given at command line.  */
+  const char *seed_help; /**< @brief Seed used to select address permutation help description.  */
+  int sender_threads_arg;	/**< @brief Threads used to send packets (default='1').  */
+  char * sender_threads_orig;	/**< @brief Threads used to send packets original value given at command line.  */
+  const char *sender_threads_help; /**< @brief Threads used to send packets help description.  */
+  int probes_arg;	/**< @brief Number of probes to send to each IP (default='1').  */
+  char * probes_orig;	/**< @brief Number of probes to send to each IP original value given at command line.  */
+  const char *probes_help; /**< @brief Number of probes to send to each IP help description.  */
+  const char *dryrun_help; /**< @brief Don't actually send packets help description.  */
+  char * source_port_arg;	/**< @brief Source port(s) for scan packets.  */
+  char * source_port_orig;	/**< @brief Source port(s) for scan packets original value given at command line.  */
+  const char *source_port_help; /**< @brief Source port(s) for scan packets help description.  */
+  char * source_ip_arg;	/**< @brief Source address(es) for scan packets.  */
+  char * source_ip_orig;	/**< @brief Source address(es) for scan packets original value given at command line.  */
+  const char *source_ip_help; /**< @brief Source address(es) for scan packets help description.  */
+  char * gateway_mac_arg;	/**< @brief Specify gateway MAC address.  */
+  char * gateway_mac_orig;	/**< @brief Specify gateway MAC address original value given at command line.  */
+  const char *gateway_mac_help; /**< @brief Specify gateway MAC address help description.  */
+  char * interface_arg;	/**< @brief Specify network interface to use.  */
+  char * interface_orig;	/**< @brief Specify network interface to use original value given at command line.  */
+  const char *interface_help; /**< @brief Specify network interface to use help description.  */
+  char * probe_module_arg;	/**< @brief Select probe module (default='tcp_synscan').  */
+  char * probe_module_orig;	/**< @brief Select probe module original value given at command line.  */
+  const char *probe_module_help; /**< @brief Select probe module help description.  */
+  char * output_module_arg;	/**< @brief Select output module (default='simple_file').  */
+  char * output_module_orig;	/**< @brief Select output module original value given at command line.  */
+  const char *output_module_help; /**< @brief Select output module help description.  */
+  char * probe_args_arg;	/**< @brief Arguments to pass to probe module.  */
+  char * probe_args_orig;	/**< @brief Arguments to pass to probe module original value given at command line.  */
+  const char *probe_args_help; /**< @brief Arguments to pass to probe module help description.  */
+  char * output_args_arg;	/**< @brief Arguments to pass to output module.  */
+  char * output_args_orig;	/**< @brief Arguments to pass to output module original value given at command line.  */
+  const char *output_args_help; /**< @brief Arguments to pass to output module help description.  */
+  const char *list_output_modules_help; /**< @brief List available output modules help description.  */
+  const char *list_probe_modules_help; /**< @brief List available probe modules help description.  */
+  char * config_arg;	/**< @brief Read a configuration file, which can specify any of these options (default='/etc/zmap/zmap.conf').  */
+  char * config_orig;	/**< @brief Read a configuration file, which can specify any of these options original value given at command line.  */
+  const char *config_help; /**< @brief Read a configuration file, which can specify any of these options help description.  */
+  const char *quiet_help; /**< @brief Do not print status updates help description.  */
+  const char *summary_help; /**< @brief Print configuration and summary at end of scan help description.  */
+  int verbosity_arg;	/**< @brief Level of log detail (0-5) (default='3').  */
+  char * verbosity_orig;	/**< @brief Level of log detail (0-5) original value given at command line.  */
+  const char *verbosity_help; /**< @brief Level of log detail (0-5) help description.  */
+  const char *help_help; /**< @brief Print help and exit help description.  */
+  const char *version_help; /**< @brief Print version and exit help description.  */
+  
+  unsigned int target_port_given ;	/**< @brief Whether target-port was given.  */
+  unsigned int output_file_given ;	/**< @brief Whether output-file was given.  */
+  unsigned int blacklist_file_given ;	/**< @brief Whether blacklist-file was given.  */
+  unsigned int whitelist_file_given ;	/**< @brief Whether whitelist-file was given.  */
+  unsigned int max_targets_given ;	/**< @brief Whether max-targets was given.  */
+  unsigned int max_results_given ;	/**< @brief Whether max-results was given.  */
+  unsigned int max_runtime_given ;	/**< @brief Whether max-runtime was given.  */
+  unsigned int rate_given ;	/**< @brief Whether rate was given.  */
+  unsigned int bandwidth_given ;	/**< @brief Whether bandwidth was given.  */
+  unsigned int cooldown_time_given ;	/**< @brief Whether cooldown-time was given.  */
+  unsigned int seed_given ;	/**< @brief Whether seed was given.  */
+  unsigned int sender_threads_given ;	/**< @brief Whether sender-threads was given.  */
+  unsigned int probes_given ;	/**< @brief Whether probes was given.  */
+  unsigned int dryrun_given ;	/**< @brief Whether dryrun was given.  */
+  unsigned int source_port_given ;	/**< @brief Whether source-port was given.  */
+  unsigned int source_ip_given ;	/**< @brief Whether source-ip was given.  */
+  unsigned int gateway_mac_given ;	/**< @brief Whether gateway-mac was given.  */
+  unsigned int interface_given ;	/**< @brief Whether interface was given.  */
+  unsigned int probe_module_given ;	/**< @brief Whether probe-module was given.  */
+  unsigned int output_module_given ;	/**< @brief Whether output-module was given.  */
+  unsigned int probe_args_given ;	/**< @brief Whether probe-args was given.  */
+  unsigned int output_args_given ;	/**< @brief Whether output-args was given.  */
+  unsigned int list_output_modules_given ;	/**< @brief Whether list-output-modules was given.  */
+  unsigned int list_probe_modules_given ;	/**< @brief Whether list-probe-modules was given.  */
+  unsigned int config_given ;	/**< @brief Whether config was given.  */
+  unsigned int quiet_given ;	/**< @brief Whether quiet was given.  */
+  unsigned int summary_given ;	/**< @brief Whether summary was given.  */
+  unsigned int verbosity_given ;	/**< @brief Whether verbosity was given.  */
+  unsigned int help_given ;	/**< @brief Whether help was given.  */
+  unsigned int version_given ;	/**< @brief Whether version was given.  */
+
+} ;
+
+/** @brief The additional parameters to pass to parser functions */
+struct cmdline_parser_params
+{
+  int override; /**< @brief whether to override possibly already present options (default 0) */
+  int initialize; /**< @brief whether to initialize the option structure gengetopt_args_info (default 1) */
+  int check_required; /**< @brief whether to check that all required options were provided (default 1) */
+  int check_ambiguity; /**< @brief whether to check for options already specified in the option structure gengetopt_args_info (default 0) */
+  int print_errors; /**< @brief whether getopt_long should print an error message for a bad option (default 1) */
+} ;
+
+/** @brief the purpose string of the program */
+extern const char *gengetopt_args_info_purpose;
+/** @brief the usage string of the program */
+extern const char *gengetopt_args_info_usage;
+/** @brief all the lines making the help output */
+extern const char *gengetopt_args_info_help[];
+
+/**
+ * The command line parser
+ * @param argc the number of command line options
+ * @param argv the command line options
+ * @param args_info the structure where option information will be stored
+ * @return 0 if everything went fine, NON 0 if an error took place
+ */
+int cmdline_parser (int argc, char **argv,
+  struct gengetopt_args_info *args_info);
+
+/**
+ * The command line parser (version with additional parameters - deprecated)
+ * @param argc the number of command line options
+ * @param argv the command line options
+ * @param args_info the structure where option information will be stored
+ * @param override whether to override possibly already present options
+ * @param initialize whether to initialize the option structure my_args_info
+ * @param check_required whether to check that all required options were provided
+ * @return 0 if everything went fine, NON 0 if an error took place
+ * @deprecated use cmdline_parser_ext() instead
+ */
+int cmdline_parser2 (int argc, char **argv,
+  struct gengetopt_args_info *args_info,
+  int override, int initialize, int check_required);
+
+/**
+ * The command line parser (version with additional parameters)
+ * @param argc the number of command line options
+ * @param argv the command line options
+ * @param args_info the structure where option information will be stored
+ * @param params additional parameters for the parser
+ * @return 0 if everything went fine, NON 0 if an error took place
+ */
+int cmdline_parser_ext (int argc, char **argv,
+  struct gengetopt_args_info *args_info,
+  struct cmdline_parser_params *params);
+
+/**
+ * Save the contents of the option struct into an already open FILE stream.
+ * @param outfile the stream where to dump options
+ * @param args_info the option struct to dump
+ * @return 0 if everything went fine, NON 0 if an error took place
+ */
+int cmdline_parser_dump(FILE *outfile,
+  struct gengetopt_args_info *args_info);
+
+/**
+ * Save the contents of the option struct into a (text) file.
+ * This file can be read by the config file parser (if generated by gengetopt)
+ * @param filename the file where to save
+ * @param args_info the option struct to save
+ * @return 0 if everything went fine, NON 0 if an error took place
+ */
+int cmdline_parser_file_save(const char *filename,
+  struct gengetopt_args_info *args_info);
+
+/**
+ * Print the help
+ */
+void cmdline_parser_print_help(void);
+/**
+ * Print the version
+ */
+void cmdline_parser_print_version(void);
+
+/**
+ * Initializes all the fields a cmdline_parser_params structure 
+ * to their default values
+ * @param params the structure to initialize
+ */
+void cmdline_parser_params_init(struct cmdline_parser_params *params);
+
+/**
+ * Allocates dynamically a cmdline_parser_params structure and initializes
+ * all its fields to their default values
+ * @return the created and initialized cmdline_parser_params structure
+ */
+struct cmdline_parser_params *cmdline_parser_params_create(void);
+
+/**
+ * Initializes the passed gengetopt_args_info structure's fields
+ * (also set default values for options that have a default)
+ * @param args_info the structure to initialize
+ */
+void cmdline_parser_init (struct gengetopt_args_info *args_info);
+/**
+ * Deallocates the string fields of the gengetopt_args_info structure
+ * (but does not deallocate the structure itself)
+ * @param args_info the structure to deallocate
+ */
+void cmdline_parser_free (struct gengetopt_args_info *args_info);
+
+/**
+ * The config file parser (deprecated version)
+ * @param filename the name of the config file
+ * @param args_info the structure where option information will be stored
+ * @param override whether to override possibly already present options
+ * @param initialize whether to initialize the option structure my_args_info
+ * @param check_required whether to check that all required options were provided
+ * @return 0 if everything went fine, NON 0 if an error took place
+ * @deprecated use cmdline_parser_config_file() instead
+ */
+int cmdline_parser_configfile (const char *filename,
+  struct gengetopt_args_info *args_info,
+  int override, int initialize, int check_required);
+
+/**
+ * The config file parser
+ * @param filename the name of the config file
+ * @param args_info the structure where option information will be stored
+ * @param params additional parameters for the parser
+ * @return 0 if everything went fine, NON 0 if an error took place
+ */
+int cmdline_parser_config_file (const char *filename,
+  struct gengetopt_args_info *args_info,
+  struct cmdline_parser_params *params);
+
+/**
+ * Checks that all the required options were specified
+ * @param args_info the structure to check
+ * @param prog_name the name of the program that will be used to print
+ *   possible errors
+ * @return
+ */
+int cmdline_parser_required (struct gengetopt_args_info *args_info,
+  const char *prog_name);
+
+
+#ifdef __cplusplus
+}
+#endif /* __cplusplus */
+#endif /* ZOPT_H */