Jon Parise's Blog, page 3

Reloading Python Modules

Being able to reload code modules is one of the many nice features of Python.
This allows developers to modify parts of a Python application while the
interpreter is running. In general, all that needs to be done is pass a
module object to the imp.reload() function (or just reload() in Python
2.x), and the module will be reloaded from its source file.

There are a few potential complications, however.

If any other code references symbols exported by the reloaded module, they maystill be bound t...

Password Composer for iPhone

I often use Password Composer (written by Johannes la Poutré) to generate
unique, per-site passwords. It does an excellent job because it's simple,
unobtrusive, and reliable. The one downside is that you need to have it
available in order to (re)generate the password for a given web site, and that
isn't always convenient, despite the large number of existing Password
Composer implementations.

The main place I find myself missing Password Composer is on my iPhone. Theonly current solution...

Android Text Links Using Linkify

The Android framework provides an easy way to automatically convert text
patterns into clickable links. By default, Android knows how to recognize web
URLs, email addresses, map addresses, and phone numbers, but it also includes
a flexible mechanism for recognizing and converting additional text patterns,
as well.

The Android Developers Blog has an article entitled Linkify your Text!that provides a nice overview of the system. It discusses how the Linkifyclass can be used to enable the...

Android Text Links Using Linkify

The Android framework provides an easy way to automatically convert text
patterns into clickable links. By default, Android knows how to recognize web
URLs, email addresses, map addresses, and phone numbers, but it also includes
a flexible mechanism for recognizing and converting additional text patterns,
as well.

The Android Developers Blog has an article entitled Linkify your
Text! that provides a nice overview of the system. It discusses how the
Linkify class can be used to enable the default link patterns and
then continues with a more advanced WikiWords example that demonstrates
custom links. That article is a fine introduction to the system, so the rest
of this article will primarily focus on details not covered therein.

All of the examples in this article are based on the TextView widget. The
Linkify class can also be used to add links to Spannable text, but those
use cases won���t be covered here because their usage is nearly identical to the
TextView cases.

TextView AutoLinking

The TextView widget features an android:autoLink attribute that
controls the types of text patterns that are automatically recognized and
converted to clickable links. This attribute is a convenient way to enable
one or more of the default link patterns because it can be configured directly
from a layout without involving any additional code.

However, for those cases where programmatically setting this value is useful,
the setAutoLinkMask() function exists.

There is one important caveat to using this ���auto-linking��� functionality,
however. It appears that when ���auto-linking��� is enabled, all additional
Linkify operations are ignored. It���s unclear whether this behavior is
intentional or inadvertent, so it���s possible things could change in future
released of the Android SDK. Consider disabling ���auto-linking��� before using
any of the Linkify operations discussed below.

// Disable the text view's auto-linking behavior
textView.setAutoLinkMask(0);

Default Link Patterns

Enabling support for one of Android���s default link patterns is very easy.
Simply use the addLinks(TextView text, int mask) function and
specify a mask that describes the desired link types.

import android.text.util.Linkify;

// Recognize phone numbers and web URLs
Linkify.addLinks(text, Linkify.PHONE_NUMBERS | Linkify.WEB_URLS);

// Recognize all of the default link text patterns
Linkify.addLinks(text, Linkify.ALL);

// Disable all default link detection
Linkify.addLinks(text, 0);

Custom Link Patterns

Detecting additional types of link patterns is easy, too. The
addLinks(TextView text, Pattern pattern, String scheme)
function detects links based on a regular expression pattern.

import java.util.regex.Pattern;
import android.text.util.Linkify;

// Detect US postal ZIP codes and link to a lookup service
Pattern pattern = Pattern.compile("\\d{5}([\\-]\\d{4})?");
String scheme = "http://zipinfo.com/cgi-local/zipsrch....
Linkify.addLinks(text, pattern, scheme);

The text is scanned for pattern matches. Matches are converted to links that
are generated by appending the matched text to the provided URL scheme base.

Note that the scheme doesn���t have to be an external web-like URL. It could
also be an Android Content URI that can be used in conjunction with a content
provider to reference application resources, for example.

Match Filters

Regular expressions are a very powerful way to match text patterns, but
sometimes a bit more flexibility is needed. The MatchFilter class
provides this capability by giving user code a chance to evaluate the link
worthiness of some matched text.

import java.util.regex.Pattern;
import android.text.util.Linkify;
import android.text.util.Linkify.MatchFilter;

// A match filter that only accepts odd numbers.
MatchFilter oddFilter = new MatchFilter() {
public final boolean acceptMatch(CharSequence s, int start, int end) {
int n = Character.digit(s.charAt(end-1), 10);
return (n & 1) == 1;
}
};

// Match all digits in the pattern but restrict links to only odd
// numbers using the filter.
Pattern pattern = Pattern.compile("[0-9]+");
Linkify.addLinks(text, pattern, "http://...", oddFilter, null);

A more complex (but useful!) example would involve matching valid dates. The
regular expression could be generous enough to match strings like ���2010-02-30���
(February 30, 2010), but a match filter could provide the logic to reject
bogus calendar dates.

Transform Filters

Up until this point, the final link was always being generated based on the
exact matched text. There are many cases where that is not desirable,
however. For example, it���s common to mention a username using the @username
syntax, but the resulting link should only include the username portion of
the text. The TransformFilter class provides a solution.

import java.util.regex.Pattern;
import android.text.util.Linkify;
import android.text.util.Linkify.TransformFilter;

// A transform filter that simply returns just the text captured by the
// first regular expression group.
TransformFilter mentionFilter = new TransformFilter() {
public final String transformUrl(final Matcher match, String url) {
return match.group(1);
}
};

// Match @mentions and capture just the username portion of the text.
Pattern pattern = Pattern.compile("@([A-Za-z0-9_-]+)");
String scheme = "http://twitter.com/";
Linkify.addLinks(text, pattern, scheme, null, mentionFilter);

This approach uses the regular expression���s capture syntax to extract just the
username portion of the pattern as a uniquely addressable match group.
Alternatively, the transform filter could just return all of the matched text
after the first character (@), but the above approach is nice because it
keeps all of the pattern���s details within the regular expression.

Of course, transform filters can be combined with match filters for ultimate
flexibility. The Android SDK uses this approach to detect wide ranges of
phone number formats (many of which include various parentheses and dashes)
while always generating a simplified link containing only digits.

Further Reading

For more information about the specific implementation details of Android���s
link generation system, the best reference is actually the source code itself.
In addition to being a good resource for understanding the system, it���s also
the best way to track down potential bugs or misunderstandings about how the
system is intended to be used.

Linkify.java - The Linkify class itself, including the MatchFilter
and TransformFilter implementations for the standard link types.
Regex.java - A collection of regular expressions and utility functions
used by Linkify to work with the standard link types.

Password Composer for iPhone

I often use Password Composer (written by Johannes la Poutr��)
to generate unique, per-site passwords. It does an excellent job because it���s
simple, unobtrusive, and reliable. The one downside is that you need to have
it available in order to (re)generate the password for a given web site, and
that isn���t always convenient, despite the large number of existing Password
Composer implementations.

The main place I find myself missing Password Composer is on my iPhone. The
only current solution that works from Mobile Safari is the public web-based
interface, and I���m security-conscious enough to avoid using that version
whenever possible. I decided the only reasonable solution would be to find a
way to run Password Composer locally on my iPhone.

I first considered writing a full-blown native iPhone application. I know a
bit of Objective-C and have experimented with the iPhone SDK, but, additional
learning opportunities aside, this approach felt like overkill.

I then discovered the option of writing an iPhone-enhanced offline web
application. That would let me reuse Password Composer���s existing static web
form without having to work through the iPhone SDK. I found the early release
of Jonathan Stark���s Building iPhone Applications with HTML, CSS, and
JavaScript and set out to adapt Password Composer to the iPhone.

Basic Application

The basic application started as a single HTML file containing a form and some
JavaScript.

<html>
<head>
<title>Password Composer</title>
</head>
<body>

<script type="text/javascript">
// Password Composer JavaScript
</script>

<h1>Password Composer</h1>

<form>
<table>
<tr>
<td>Master Key:</td>
<td><input type="password" id="masterpwd1" onkeyup="mpwd_generate()"
onchange="mpwd_generate()"/></td>
</tr>
<tr>
<td>Domain:</td>
<td><input type="text" id="domain1" onkeyup="mpwd_generate()"
onchange="mpwd_generate()"/></td>
</tr>
<tr>
<td>Password:</td>
<td><input type="text" id="genpwd1" onkeyup="mpwd_generate()"
onchange="mpwd_generate()"/></td>
</tr>
</table>
</form>

</body>
</html>

The JavaScript and form were copied directly from Password Composer���s public
web-based interface and worked just fine in Mobile Safari.

Styling the Application

Next, I set about styling the look and feel of the page to more closely
resemble an iPhone application. I started by adding a stylesheet file. I
could have used inline CSS, and maybe I���ll merge the styles back into the main
page at some point in the future, but using a separate file keeps things more
manageable for the time being.

body {
background-color: #ddd;
color: #222;
font-family: Helvetica;
font-size: 14px;
margin: 0;
padding: 0;
}

h1 {
background-image: -webkit-gradient(linear, left top, left bottom,
from(#ccc), to(#aaa));
background-color: #ccc;
border-bottom: 1px solid #666;
color: #222;
font-size: 20px;
font-weight: bold;
padding: 10px 0;
text-align: center;
text-decoration: none;
text-shadow: 0px 1px 0px #ddd;
}

form {
background-color: #FFFFFF;
border: 1px solid #999999;
color: #222222;
font-size: 15px;
padding: 12px 10px;
margin: 10px;
-webkit-border-radius: 8px;
}

td, input {
font-size: 14px;
}

Most of these styles are influenced by examples from Jonathan Stark���s
book.

Viewport Definition

The next change is the addition of a viewport meta tag:

<meta name="viewport" content="user-scalable=no, width=device-width" />

This viewport definition convinces Mobile Safari not to rescale the HTML
content as though it were a ���normal��� 980px-wide web page.

Input Field Types

I was also interested in configuring the form���s <input> fields to pop up the
most appropriate type of on-screen keyboard on the iPhone. Master Key was
already being recognized as a password field (type=password), but I wanted
the Domain field to use the iPhone���s special URL keyboard.

A little research indicated that Mobile Safari recognizes a small
subset of the HTML5 input types, including the url type.

<input type="url" id="domain1">

Custom Icon

Because I want Password Composer to feel like a natural, first-class iPhone
application, it deserves to get a real icon. This icon is used when a link to
the application is saved to the user���s home screen. Home screen icons are
57��57 pixel PNG images.

There are two ways for a web application to specify its home screen icon:

Provide a file named apple-touch-icon.png in the site���s root.
Add a <link rel="apple-touch-icon" href="icon.png" /> tag to the page���s
<head> block.

I chose the second approach because I wasn���t planning on using an entire
domain to serve the Password Composer application. It also feels like better
encapsulation for a single-page web application.

Offline Application Cache

The last bit of work adds support for the iPhone���s offline application cache.
This cache allows the files used by a web-based applications to be stored
locally on the iPhone so that they can still be accessed when the phone
doesn���t have an active network connection. This ���offline mode��� is described
quite well in Stark���s book: Chapter 6: Going Offline.

Briefly, all that���s required is the creation of a ���manifest��� file, as
specified by HTML5. My manifest file is named pwdcomposer.manifest:

CACHE MANIFEST
index.html
iphone.css
icon.png

The manifest lists all of the files used by the web application. Manifest
files can be much more involved than this example. They can specify fallback
images for use when network-based files are inaccessible, for example. But
for my current purposes, the simple manifest above works great.

Manifest files must be served using the text/cache-manifest MIME type.
For Apache-like web servers, this can be configured using a directive like:

AddType text/cache-manifest .manifest

Lastly, the web page needs to specify its associated manifest file using the
manifest attribute of the <html> element:

<html manifest="pwdcomposer.manifest">

Deployment

All that is left to do is deploy the application. Simply serve up the source
files using a web server, and iPhone users can choose to bookmark the
application���s link just like they would for any other web page. The offline
application caching system will work automatically. When the iPhone is
connected to a network, Mobile Safari will attempt to fetch the original
manifest file to check for updates, but the application will otherwise be
running entirely from the cache, essentially making it a local iPhone
application.

My version is deployed here: https://www.indelible.org/pwdcomposer/

The complete source code is available on GitHub.

Vim Color Schemes

The Vim text editor supports highly-configurable color schemes which build
upon the editor���s rich syntax highlighting system. The stock Vim distribution
includes a number of color schemes, and many more are available from the Vim
Scripts repository.

Color scheme definitions are simply normal Vim scripts that live in the
colors/ directory of the Vim runtime hierarchy (see :help runtimepath).

Color schemes are loaded using the :colorscheme command. The scheme���s name
is determined by the filename of its script file (minus the .vim extension).
For example, to load the stock blue color scheme (which is defined by the
colors/blue.vim script):

:colorscheme blue

Creating Color Schemes

Creating a custom color scheme is quite easy. Start by creating a new Vim
script file in the colors/ directory based on the name of the new scheme.
Start the script with the following commands:

set background=dark
"or background=light
highlight clear
if exists("syntax_on")
syntax reset
endif
let g:colors_name = "example"

These commands will reset the syntax highlighting system to its default state.
Note that some color scheme scripts might prefer a light background, so that
first line should be changed accordingly. (highlight clear uses the
background value, so background must be set first.)

The final line sets the global colors_name variable to the scheme���s name
(example, in this example).

The rest of the script defines the color scheme itself. This is accomplished
primarily through the highlight (or hi) command. Each highlight command
sets the colors for a single syntax group. Setting the colors for the
Comments group might look like this:

hi Comment ctermbg=black ctermfg=darkgrey guibg=#000000 guifg=#777777

To see the full list of Vim���s syntax groups (along with their current
highlight settings), run the following command from within the editor:

:source $VIMRUNTIME/syntax/hitest.vim

Testing Runtime Features

Because the color scheme is simply a Vim script, you can conditionalize the
definitions based on various runtime values. The presence of the
gui_running feature indicates that the Vim GUI is running, for example:

if has('gui_running')
" GUI colors
else
" Non-GUI (terminal) colors
endif

And the terminal���s color range ��� the number of available colors ��� can be
queried via the &t_Co variable:

if &t_Co > 255
" More than 256 colors are available
endif

Additional Configuration

Color scheme scripts can support basic configuration using global variables.

if exists("g:example_force_dark")
set background=dark
endif

The user should set this variable in his .vimrc file before loading the
color scheme script.

let g:example_force_dark = 1
colorscheme example

(Global variables can be ���unset��� using the unlet command.)

Virtual Ethernet Tunneling

This paper discusses the implementation of virtual Ethernet tunnels using
OpenBSD. The current release of OpenBSD at the time of writing (2001) was
version 2.9, so some of the material may be fairly dated. I haven���t revisited
the details since then.

Overview

Without going too deep into the technical details, a virtual Ethernet tunnel
uses packet encapsulation, Ethernet bridging, and IPSec encryption to
tunnel a subnet from one host to another host over a public network
(generally, the Internet).

The best way to explain how this works is by describing each of the pieces
involved.

We���ll start with packet encapsulation. Ethernet frames can be encapsulation
within an IP packet and transported to a remote host, who can then strip off
the surrounding IP headers and retrieve the original, unmodified Ethernet
frame. This is the essence of tunneling packets from one network to another
over an intermediate network. OpenBSD supports this type of encapsulated
tunnel using the gif interface.

Bridging is definitely not a new networking technology, but once the OpenBSD
code was expanded to allow the inclusion of virtual interfaces in bridge
groups along with traditional physical interfaces, it opened up a number of
new possibilities. In our case, it allows us to group the generic gif
interfaces (which represent our Ethernet tunnel, as previously described) with
a physical interface. This allows the physical interfaces to share traffic
with the tunnel interface as though they existing on the same subnet. And
because the tunnel interface is really a stream of encapsulated packets from
some other host, we can effectively link two segments of the same subnet using
two bridges (one on each host) and an encapsulated tunnel to span the hosts.

(bridged interfaces) (bridged interfaces)
| | | |
v +--------+ v v +--------+ v
/----| Host 1 |-----[ public network ]-----| Host 2 |----\
| +--------+ ^ <-(gif tunnel)-> ^ +--------+ |
private | | private
network gif gif network

The final piece of the system is IPSec. While not absolutely required to
tunnel between our bridged interfaces, it adds a welcome layer of security and
helps maintain the privacy of our ���private��� network. The IPSec part of the
equation is perhaps the most complex and difficult to set up, but it scarcely
causes problems once it���s running, and it���s not really as hard to get going as
it may sound.

Required sysctl Settings

Before your system can forward packets between interfaces or handle the IPSec
protocols, the following sysctls need to be enabled (i.e. set to 1):

net.inet.ip.forwarding
net.inet.esp.enable
net.inet.ah.enable
net.inet.etherip.allow

OpenBSD can enable these settings on boot based on the contents of
/etc/sysctl.conf. Make sure you have at least the following lines
uncommented:

net.inet.ip.forwarding=1 # 1=Permit forwarding (routing) of packets
net.inet.esp.enable=1 # 1=Enable the ESP IPSec protocol
net.inet.ah.enable=1 # 1=Enable the AH IPSec protocol

These parameters can also be modified at runtime using the sysctl tool.

ipsecadm and Security Associations

OpenBSD includes a tool named ipsecadm that is used for managing the
system���s security associations (SA���s) and flows. The tool accepts a number of
arguments, but we���ll mainly be working with the SA creation syntax:

ipsecadm new esp -spi 2000 -dst 66.24.105.57 -src 129.21.111.216 -enc 3des \
-auth sha1 -key d09fffc3ebaee12362d65b38068dd381df89e4961ed282b3 -authkey \
5ee0fc2cc2197fe24417934cac6db483b53eace3

This command will create a new security association using the esp IPSec
protocol. We���ve assigned this entry an SPI (unique index) of 2000. We also
set the source and destination addresses for this SA. Note that these are
specific to the SA and don���t assume anything about the host, meaning that the
source address isn���t necessarily the local machine���s address.

We continue by defining the desired encryption algorithm (3des), the
authentication algorithm, and the two encryption keys. We���ll cover key
generation in the Generating Keys section below.

The following is an example of a second, complementary SA:

ipsecadm new esp -spi 2001 -dst 129.21.111.216 -src 66.24.105.57 -enc 3des \
-auth sha1 -key d09fffc3ebaee12362d65b38068dd381df89e4961ed282b3 -authkey \
5ee0fc2cc2197fe24417934cac6db483b53eace3

This entry is nearly the same as the previous SA. Note, however, that we���ve
assigned it a different SPI (2001). We���ve also swapped the source and
destination addresses.

We now have two SA���s defined, one for each direction of packet flow. These
two SA���s must be defined identically on both hosts ��� don���t change the
ordering of the addresses or alter the encryption protocols or keys on one of
the hosts! Because the SPI is included with each encrypted packet - and each
host uses the SPI to determine how the packet should be routed, encrypted, or
decrypted - the ordering must remain consistent. In other words, security
associations are unique to the connection, not the hosts.

The last thing we need to do is define a flow. A flow determines what
security parameters a packet should have for either input or output. Here���s
an example:

ipsecadm flow -dst 66.24.105.57 -out -transport etherip -require -addr \
129.21.111.216 255.255.255.255 66.24.105.57 255.255.255.255

This command defines a flow for packets destined for 66.24.105.57. We specify
that we���ll be transporting these packets using ���Ethernet in IP��� encapsulation,
and the packets will be traveling from us at 129.21.111.216 to our destination
at 66.24.105.57.

Unlike security associations, flows are unique to the individual host���s
configuration. The command for the opposite host would be:

ipsecadm flow -dst 129.21.111.216 -out -transport etherip -require -addr \
66.24.105.57 255.255.255.255 129.21.111.216 255.255.255.255

Note that all we���ve done here is change the flow���s destination address and
swap the source and destination address of the connection.

Generating Keys

The key���s size depends on the encryption protocol. DES and 3DES use 8 and 24
bits, respectively. The sizes for the other protocols (Rijndael, Blowfish,
CAST) may vary.

Keys can be generated using the following command:

openssl rand 24 | hexdump -e '24/1 "%02x"' && echo ""

This will generate a hexadecimal representation of a 24-bit key. If a
different key size is needed, replace the occurrences of 24 with the desired
bit size.

Configuring the Ethernet Interfaces

Two Ethernet interfaces on each host are required for this sort of tunneling.
Each interface must sit on a different subnet. One of those subnets should
obviously be the one whose addresses you want to tunnel. In our example, that
network is 129.21.60.0/24. The other network can be pretty much anything (a
cable modem network, for example).

The easiest way to configure the network interfaces is by using the
/etc/hostname.interface file. OpenBSD will execute the interface
configuration commands listed inside this file upon boot, which is generally
desirable for this kind of setup.

We���ll begin by configuring the ���public��� interface (/etc/hostname.sis0, in
my case):

inet 129.21.111.216 255.255.255.128 NONE

Be sure to set up this interface correctly (including the netmask!). In this
case, the interface sits on a nine-bit subnet, so we set the netmask
accordingly. Test that this interface works correctly before proceeding.

Now we���ll configure the second network interface, the one that sits on the
subnet that we want to tunnel (/etc/hostname.dc0, for me):

inet 129.21.60.107 255.255.255.0 NONE


A Note on Gateway Addresses

It is important to note that each host���s gateway address must be set to the
���public��� network���s gateway (not the subnet that you are tunneling!). Things
will not work correctly otherwise.

The system���s gateway address can be statically configured using the
/etc/mygate file. It might contain a line like this:

129.21.111.254

If your host uses DHCP to configure its gateway address (in the case of a
cable modem provider), this will all be handled for you and there is no need
to configure your gateway address by hand.

The Generic Interface

The generic interface (gif) is used for the actual tunneling between the
hosts. This interface is purely virtual, meaning it is not necessarily bound
to any physical interface on the system. Instead, it is given a source and a
destination address between which to tunnel its encapsulated by packets.

More detailed information is available in the manpage.

The gif interface can be configured at boot via the /etc/hostname.gif0
file. Here is the /etc/hostname.gif0 file that I use:

giftunnel 129.21.111.216 66.24.105.57
up

The first line establishes the tunnel between the local (source) address and
the remote (destination) address. The second line activates the interface.

Both of these command strings are passed directly to ifconfig.

The Bridge Interface

OpenBSD includes excellent Ethernet bridging support. Each bridge is
represented by a bridge interface (e.g. /dev/bridge0). Bridge
configuration is performed using the brconfig tool.

Each bridge can have an arbitrary number of interfaces added to it. These
interfaces can either be physical network interfaces or virtual encapsulation
interfaces (such as the gif interface).

More detailed information is available in the manpage.

Bridge configuration can also be performed upon boot. This is accomplished
through the /etc/bridgename.bridge0 file. Here���s mine:

add gif0
add dc0
#
!ipsecadm flush
!ipsecadm new esp -spi 2000 -dst 66.24.105.57 -src 129.21.111.216 -enc 3des \
-auth sha1 -key d09fffc3ebaee12362d65b38068dd381df89e4961ed282b3 -authkey \
5ee0fc2cc2197fe24417934cac6db483b53eace3
!ipsecadm new esp -spi 2001 -dst 129.21.111.216 -src 66.24.105.57 -enc 3des \
-auth sha1 -key d09fffc3ebaee12362d65b38068dd381df89e4961ed282b3 -authkey \
5ee0fc2cc2197fe24417934cac6db483b53eace3
!ipsecadm flow -dst 66.24.105.57 -out -transport etherip -require -addr \
129.21.111.216 255.255.255.255 66.24.105.57 255.255.255.255
#
up

The contents of this file will require a little bit of explanation.

The first two lines add the gif0 and dc0 interfaces to this bridge
interface. Once these two interfaces are placed in a bridge group, packets
will be able to move freely between them, as if they existing on the same
physical Ethernet segment.

The second set of commands are all prefixed with a exclamation point (!).
This indicates that these commands are not ifconfig parameters. Instead,
they should be executed on their own during the configuration sequence.

These commands set up the various security associations using ipsecadm, as
was discussed earlier. Note that the first thing we do is flush any existing
security associations and flows. This starts us with a clean slate every
time.

The last line simply activates the bridge interface.

Additional Information

Using kernfs

OpenBSD���s kernfs virtual file system exports some useful information on the
system���s current IPSec settings. This information can be very handy when
debugging setups and gathering statistics.

To begin, you must first mount the kernfs file system (which isn���t done as
part of a default installation). You���ll need to create a new mount point
(/kern is typical):

# mkdir /kern

Next, you���ll need to add the following line to you /etc/fstab file:

/kern /kern kernfs ro 0 0

You can now mount the kernfs file system using the command:

# mount /kern

The information we want is stored in /kern/ipsec. It can be viewed simply
by cat���ing the file:

% cat /kern/ipsec

After you���ve set up a couple of security associations, your output will look
something like this:

Hashmask: 31, policy entries: 1
SPI = 00002000, Destination = 66.24.105.57, Sproto = 50
Established 1283589 seconds ago
Source = 129.21.111.216
Flags (00000000) =
Crypto ID: 1
xform =
Encryption = <3DES>
Authentication =
174365335 bytes processed by this SA
Expirations:
(none)

SPI = 00002001, Destination = 129.21.111.216, Sproto = 50
Established 1283589 seconds ago
Source = 66.24.105.57
Flags (00000000) =
Crypto ID: 2
xform =
Encryption = <3DES>
Authentication =
405224 bytes processed by this SA
Expirations:
(none)


Increasing the Number of Interfaces

Because each virtual Ethernet tunnel requires one gif interface and one
bridge interface, a stock OpenBSD installation can only support two tunnels
(only two bridge interfaces are available by default in OpenBSD 2.9).

Increasing the number of available is fairly trivial, however. The downside
is that it currently requires a kernel recompilation, which inherently
requires a reboot in order to see the effects of the new kernel.

Note that the details of building a kernel under OpenBSD are outside the scope
of this document, but the topic is conferred pretty well as part of the
OpenBSD FAQ.

Begin by editing your kernel configuration file. Note that the definitions
for the gif and bridge devices are in the GENERIC kernel configuration
file, which your configuration file probably includes. We���ll simply be
overriding these default entries.

Add the following lines to your kernel configuration file:

pseudo-device gif 8 # IPv[46] over IPv[46] tunnel (RFC1933)
pseudo-device bridge 8 # network bridging support

Feel free to replace the number of devices (8, in this case) with a number of
your own choosing. It���s up to you and your setup.

Continue on building your custom kernel. Note that config might complain
about the redefinition of the above interfaces, but the warning is only
informational. Your new values will override the defaults.

A reboot using your new kernel is required for the new devices to become
available.

References


ipsecadm(8) manpage
gif(4) manpage
bridge(4) manpage
ifconfig(8) manpage
brconfig(8) manpage

Classless in-addr.arpa. Delegation

Classless in-addr.arpa. delegation allows network administrators to provide
authoritative reverse DNS on subnets that don���t fall on octet boundaries.
This is especially useful for subnets comprised of less than eight bits in the
host portion of the address (i.e. smaller than a class C).

There are two important things to remember: first, we���re dealing with
classless subnets, meaning they don���t align themselves neatly with IPv4���s
octet boundaries (like a class A, B, C, D, or E network); and second, only one
name server can be the primary authoritative controller of a given class of IP
addresses.

In order for a non-octet subnet network administrator to properly provide
reverse DNS for his hosts, he will require the cooperation of his upstream
provider. This goes back to the second rule above: the upstream provider
technically controls the complete class of addresses out of which the
downstream subnet receives its address space.

I found myself in this position years ago while working for a small web-based
startup. We were allocated a block of 32 addresses as a /27 subnet (netmask:
255.255.255.224). What follows is a brief summary (with examples) of how we
went about establishing our name servers as the authorities for reverse DNS in
our zone.

All given configurations were tested using BIND 8 under both OpenBSD
and FreeBSD circa 2000, but I expect they are still relevant today.

Our assigned subnet was 206.126.7.64/27 (in CIDR notation). Thus, our
network address was 206.126.7.64 and our broadcast address was 206.126.7.95,
giving us 30 usable host addresses in between.

We begin by creating a standard zone file (example-hosts) for the
example.com domain (for ���forward��� DNS queries):

@ IN A 206.126.7.73
ns IN A 206.126.7.65
[...]
gateway IN A 206.126.7.94

There really isn���t anything special going on here. Simply add this entry to
your named.conf file:

zone "example.com" {
type master;
file "example-hosts";
};

Next, we define the reverse lookup mappings for the 206.126.7.64/27 subnet in
a file named 64-27.7.126.206.rev:

65 IN PTR ns.example.com.
[...]
94 IN PTR gateway.example.com.

Now, for this file, we add a slightly different entry to named.conf:

zone "64/27.7.126.206.in-addr.arpa" {
type slave;
file "64-27.7.126.206.rev";
masters {
206.126.7.65;
};
};

Be sure to list your provider���s name server(s) in the masters {} block. As
far as the provider is concerned, they are still the authoritative ���master��� of
this IP address zone even though they delegate the mappings within our subnet
to us. We must act as the slave and participate in zone transfers.

That���s really all the subnet administrator has to worry about. The rest of
the work needs to be done on the provider���s side. In order for them to
delegate the reverse lookups to the downstream name servers, we employ (or
perhaps abuse) the CNAME record. These entries go in a file named
7.126.206.rev. Note that this file contains entries for the entire
206.126.7.0/24 class (C) of addresses, not just our 206.126.7.64/27 subnet.

;; example.com
;
; Name servers for 206.126.7.64/27
;
64/27 NS ns.example.com. ; 206.126.7.65
64/27 NS raq.example.com. ; 206.126.7.66
;
; Classless in-addr.arpa. delegation for 206.126.7.64/27
;
65 CNAME 65.64/27.7.126.206.in-addr.arpa.
[...]
94 CNAME 94.64/27.7.126.206.in-addr.arpa.

The provider���s named.conf should contain an entry like:

zone "7.126.206.in-addr.arpa" {
type master;
file "7.126.206.rev";
};

That���s really all there is to it. It took me a couple of hours to muddle my
way through all of the details (and making a few silly mistakes along the way
didn���t help much either), but it���s really not that complicated. When a
reverse DNS query is issued, it���s sent to the authoritative name server (the
provider���s), who in turn delegates the request to the subnet���s name servers,
who then get the final say. This is transparent to the querying client, and
everyone is happy in the end.

For more (reputable) information on this topic, see RFC 2317.

You may also want to check out Avoid RFC 2317 style delegation of
���in-addr.arpa.���, which describes an alternative to RFC 2317. I offer no
opinion as to which method is the best solution for a given situation,
however.

Twisted Python and Bonjour

Bonjour (formerly Rendezvous) is Apple���s service discovery
protocol. It operates over local networks via multicast DNS. Server
processes announce their availability by broadcasting service records and
their associated ports. Clients browse the network in search of specific
service types, potentially connecting to the service on the advertised port
using the appropriate network protocol for that service.

A common example of Bonjour in action is iTunes��� music library sharing
feature. iTunes sharing uses DAAP (Digital Audio Access Protocol).
iTunes uses Bonjour to announce its local shared libraries as well as to
browse the network for remote DAAP servers.

Twisted Python, while supporting a wide range of network protocols
by default, currently lacks an official Bonjour or multicast DNS
implementation. The start of a multicast DNS implementation exists in
Itamar���s sandbox,
but it hasn���t been updated since 2004.

Given that, applications that want to use Bonjour-based service discovery must
provide their own implementation. Unfortunately, there can only be one
Bonjour ���responder��� running on a system at one time. If multiple applications
attempted to advertise services by standing up competing responders, a port
conflict would arise. Therefore, all of the Bonjour-aware applications
running on a system must coordinate.

On Mac OS 10.2 and later, applications can simply communicate with the
operating system���s built-in Bonjour service. Most other operation systems
don���t provide native Bonjour functionality, but support is generally available
via third-party packages. Apple provides Bonjour for Windows,
and the LGPL-licensed Avahi runs on most other platforms.

Supporting multiple potential Bonjour interfaces can be a burden for
application developers. Fortunately, for Python-based projects, pybonjour
exists to provide a very nice ctypes-based abstraction layer to all of the
Bonjour-compatible libraries mentioned above.

pybonjour���s public API is based on ���service descriptors���. Each operation
returns a service descriptor reference and signals the caller via a callback
when the operation completes. Service descriptors can generally be treated
like read-only file descriptors, but all read operations must be done using
pybonjour���s DNSServiceProcessResult() function.

We can easily wrap a pybonjour service descriptor in an object that implements
Twisted���s IReadDescriptor interface:

import pybonjour
from twisted.internet.interfaces import IReadDescriptor
from zope import interface

class ServiceDescriptor(object):

interface.implements(IReadDescriptor)

def __init__(self, sdref):
self.sdref = sdref

def doRead(self):
pybonjour.DNSServiceProcessResult(self.sdref)

def fileno(self):
return self.sdref.fileno()

def logPrefix(self):
return "bonjour"

def connectionLost(self, reason):
self.sdref.close()

Then, it���s simply a matter of writing some operation-specific functions that
join the pybonjour interface with Twisted���s event-driven concepts. These
functions initiate the pybonjour request, handle the pybonjour callback, and
dispatch the result using a Twisted Deferred.

The following example broadcasts a new service record. Note that the local
callback function handles both the success and error results.

from twisted.internet.defer import Deferred

def broadcast(reactor, regtype, port, name=None):
def _callback(sdref, flags, errorCode, name, regtype, domain):
if errorCode == pybonjour.kDNSServiceErr_NoError:
d.callback((sdref, name, regtype, domain))
else:
d.errback(errorCode)

d = Deferred()
sdref = pybonjour.DNSServiceRegister(name = name,
regtype = regtype,
port = port,
callBack = _callback)

reactor.addReader(ServiceDescriptor(sdref))
return d

The caller can then provide callback and errback functions that will be
invoked using Twisted���s event-based reactor machinery.

from twisted.internet import reactor
from twisted.python import log

sdref = None

def broadcasting(args):
global sdref
sdref = args[0]
log.msg('Broadcasting %s.%s%s' % args[1:])

def failed(errorCode):
log.err(errorCode)

d = broadcast(reactor, "_daap._tcp", 3689, "DAAP Server")
d.addCallback(broadcasting)
d.addErrback(failed)

To stop broadcasting, simply close the service descriptor (sdref.close()).