Recent posts (max 10) - Browse or Archive for more

Heartbleed for users

There has been a great deal of commotion about The Heartbleed Bug, particularly from the point of view of server operators. Users are being encouraged to change all their passwords, but--oh, wait--not until after the servers get fixed.

How's a poor user to know when that happens?

Well, you can base it on when the site's SSL cert was issued. If it was issued prior to the Heartbleed announcement, the keys have not been changed (but see update) in response to Heartbleed. That could be for a couple of different reasons. One is that the site was not vulnerable because it was never running a vulnerable version of OpenSSL. The other is that the site was vulnerable, and the vulnerability has been patched, but the operators of the site have not replaced their SSL keys yet.

In either of those two cases, changing your password isn't going to do much. If the site was never vulnerable, your account is not affected. If it was vulnerable, an adversary who got the private keys still has them, and changing your password does little for you.

So once a site updates its SSL cert, it then makes sense to change your password.

How do you know when that happens? Well, if you are using Firefox, you can click on the lock icon, click on the 'more information' button, then the Security tab, then the 'View Certificate' button, then look at the 'Issued On' line. Then close out that window and the previous window. ... For each site you want to check.

That got tedious.

cert_age_check.py:

#!/usr/bin/python
import sys
import ssl
import subprocess
import datetime


def check_bleeding(hostname, port):
    """Returns true if you should change your password."""
    cert = ssl.get_server_certificate((hostname, port))
    task = subprocess.Popen(['openssl', 'x509', '-text', '-noout'],
        stdin=subprocess.PIPE, stdout=subprocess.PIPE)
    readable, _ = task.communicate(cert)
    issued = [line for line in readable.splitlines() if 'Not Before' in
        line][0]
    date_string = issued.split(':', 1)[1].strip()
    issue_date = datetime.datetime.strptime(date_string,
        '%b %d %H:%M:%S %Y %Z')
    return issue_date >= datetime.datetime(2014, 4, 8, 0, 0)


def main(argv):
    """Syntax: python cert_age_check.py <hostname> [portnumber]"""
    hostname = argv[1]
    if len(argv) > 2:
        port = int(argv[2])
    else:
        # 993 and 995 matter for email...
        port = 443
    if check_bleeding(hostname, port):
        sys.stdout.write("Change your password\n")
    else:
        sys.stdout.write("Don't bother yet\n")
    return 0


if __name__ == '__main__':
    sys.exit(main(sys.argv))

This script checks the issue date of the site's SSL certificate to see if it has been issued since the Heartbleed announcement and tells you if it is time to change your password. If something goes wrong in that process, the script will fail with a traceback; I'm not attempting to make this particularly robust. (Nor, for that matter, elegant.)

If you save a list of hostnames to a file, you can run through them like this:

xargs -n 1 python cert_age_check.py < account_list

So if you have a file with

bankofamerica.com
flickr.com

you will get

Don't bother yet for bankofamerica.com
Change your password for flickr.com

While I would not suggest handing this to someone uncomfortable with a commandline, it is useful for those of us who support friends and family to quickly be able to determine what accounts to recommend they worry about and which to deal with later.

UPDATE: There is a flaw in this approach: I was surprised to learn that the cert that a CA provides to a website operator may have the same issue date as the original cert -- which makes it impossible for the user to determine if the cert is in fact new. With that wrinkle, if you are replacing your cert due to heartbleed, push your CA to give you a cert with a new issue date as evidence that you have fixed your security.

Something I mentioned elsewhere, but did not explicitly state here, is that even with a newly dated cert, a user still cannot tell if the private key was changed along with the cert. If the cert has not changed, the private key has not either. If the operator changes the cert, they will have changed the private key at that point if they are going to do so.

This gets us back to issues of trust. A security mechanism must have a way to recover from a security failure; that is widely understood. But Heartbleed is demonstrating that a security mechanism must include externally visible evidence of the recovery, or the recovery is not complete.

UPDATE: For this site, I buy my SSL cert through DreamHost. I had to open a help ticket to get them to remove the existing cert from the domain in their management application before I could get a new cert. (If you already have a valid cert, the site will let you click on buttons to buy a new cert, but it won't actually take any action on it. That is a reasonable choice in order to avoid customers buying duplicate certs -- but it would be nice to be able to do so anyway.) The response to my help ticket took longer than I would have liked, but I can understand they're likely swamped, and probably don't have a lot of automation around this since they would reasonably not foresee needing it. Once they did that, I then had to buy a new cert from them. I was happy to see that the new cert I bought is good for more than a year -- it will expire when the next cert I would have needed to buy would have expired. Which means that while I had to pay for a new cert, in the long run it will not cost me anything extra. And the new cert has an updated issue date so users can see that I have updated it.

Lego + neodymium magnets = win

Our refrigerator door is cluttered with all the usual papers and pictures of various and sundry sources. To hold all that up, the door is plagued by the litter of lousy magnets that barely win out over the inexorable pull of gravity.

No more.

Lego magnets

I bought a selection of neodymium magnets in sizes that fit Lego plates and bricks.

Gluing a single magnet into the base of a Lego piece can be tricky. You must work with a single magnet at a time, or they have a tendency to jump to each other, making the glue you just put on them go places you don't want it to. I found that to keep the glued pieces from jumping to each other and making a gooey mess, I needed to build a frame that I could connect the glued pieces to. That allowed me to work on gluing more magnets while the others dried.

Since the void inside the Lego piece is deeper than the magnet by a little bit, you need to make sure the magnet is flush with the bottom of the Lego piece. You can do that by setting the magnet on the work surface, applying glue to it, and then carefully placing the Lego piece over it. And since you don't want to glue these things to your counter, you need to do all of this on a piece of wax paper.

Another pitfall to avoid is applying glue around all sides of the magnet. When gluing a 1x1 brick, I found that air was trapped behind the magnet, so I would press the magnet into the brick and the air would push it half way out again. Applying glue to the magnet so that one side isn't glued allows that air to escape so you don't have to fight Boyle's law.

The result was surprisingly strong. The plates can hold photos easily, and the bricks are a lot stronger than that. A single magnet in a plate holds a lego mosaic made of a couple dozen parts to the refrigerator quite easily.

And now colorful Lego pieces cling tenaciously to the refrigerator.

Better line wrapping in Vim, 8th iteration

Here is the breakindent patch updated for Vim 7.4.16 from Fedora 19: vim-7.4.16-fc19-breakindent.patch

Update: This also applies cleanly to Fedora 19's vim-7.4.027-2.

  • Posted: 2013-09-12 18:53 (Updated: 2013-09-14 11:37)
  • Author: retracile
  • Categories: vim
  • Comments (0)

Subtleties of colorizing unified diff output

I wanted to colorize unified diff output on the commandline; red for deletions and green for additions. As I learned on StackOverflow, there is colordiff which is the right solution for the problem. But why use an existing solution written by someone else in 400 lines of Perl, when you can use a partial solution of your own in one line of sed?

Wait... don't answer that.

So here's a one-liner that highlights deletions with a red background and additions with a green background, and hunk markers in blue text.

sed 's/^-/\x1b[41m-/;s/^+/\x1b[42m+/;s/^@/\x1b[34m@/;s/$/\x1b[0m/'

I chose to highlight changes using a background color so that whitespace changes would be more readily apparent. Interestingly, xterm does not display a background color for tab characters. This means that you are able to clearly see tab <-> space indentation changes in a diff. However, it also means that you can't see changes of trailing tabs. Sadly, colordiff does not support background colors.

Filenames are highlighted in the same way as content... for a good reason. You see, to differentiate between a filename line and a content line, you have to fully parse the diff output. Otherwise, if you add a line of text to a file that looks like:

++ vim73/src/ui.c      2013-06-01 14:48:45.012467754 -0500

you will get a line in your unified diff that looks like:

+++ vim73/src/ui.c      2013-06-01 14:48:45.012467754 -0500

which any regex-based approach is going to incorrectly see as a diff filename header. Clearly the same problem arises when deleting lines that start with --. Since colordiff is also a line-by-line regex-based implementation, it also highlights filenames the same as content. This is one of those cases where you can change your problem specification to make your solution trivial.

Example:

  • evil.orig
    blah blah blah
    humbug
    one two three
    four five six
    -- vim73/src/ui.c      2013-06-01 14:48:45.012467754 -0500
    @@ -1,6 +1,6 @@
    blah blah blah
    one two three
    four five six
    eight nine ten
    zero
    humbug
    
  • evil.new
    blah blah blah
    bah humbug
    one two three
    four five six
    ++ vim73/src/ui.c      2013-06-01 14:48:45.012467754 -0500
    @@ -1,6 +1,6 @@
    blah blah blah
    one two three
    four five six
    seven eight nine ten
    zero
    humbug
    

Yields a misleading unified diff that looks like:

--- evil.orig   2013-06-01 16:18:25.282693446 -0500
+++ evil.new    2013-06-01 16:30:27.535803954 -0500
@@ -1,12 +1,12 @@
 blah blah blah
-humbug
+bah humbug
 one two three
 four five six
--- vim73/src/ui.c      2013-06-01 14:48:45.012467754 -0500
+++ vim73/src/ui.c      2013-06-01 14:48:45.012467754 -0500
 @@ -1,6 +1,6 @@
 blah blah blah
 one two three
 four five six
-eight nine ten
+seven eight nine ten
 zero
 humbug

That one space before the false hunk header is probably the most visually apparent clue that something isn't right. Unless you're paying attention to the actual numbers in the hunk header, that is; but if the hunk is a couple hundred lines long and the false diff portion is only a couple of lines, even that would be hard to notice.

Colorize the diff (with my sed implementation),

--- evil.orig   2013-06-01 16:18:25.282693446 -0500
+++ evil.new    2013-06-01 16:30:27.535803954 -0500
@@ -1,12 +1,12 @@
 blah blah blah
-humbug
+bah humbug
 one two three
 four five six
--- vim73/src/ui.c      2013-06-01 14:48:45.012467754 -0500
+++ vim73/src/ui.c      2013-06-01 14:48:45.012467754 -0500
 @@ -1,6 +1,6 @@
 blah blah blah
 one two three
 four five six
-eight nine ten
+seven eight nine ten
 zero
 humbug

... and it is slightly less subtle.

Perhaps there is a case here for a diff colorizer built on a real parse of a unified diff?

Better line wrapping in Vim, 7th iteration

Here is the breakindent patch updated for Vim 7.3.944 from Fedora 17: vim-7.3.944-fc17-breakindent.patch

Buffalo DD-WRT OpenVPN netmask bug

In the hope that I can save others the bit of frustration I recently went through, I wanted to describe a bug in the DD-WRT firmware shipped on Buffalo's "AirStation HighPower N450 Giga" (model WZR-HP-G450H). DD-WRT has already fixed the problem, and I found a workaround to help others who run into the same problem.

The bug:

You can't set the VPN netmask when setting up a "Router (TUN)" VPN. No matter what you enter, it always comes back with "0.0.0.0"

The fix:

This was fixed in changeset 20392. But Buffalo ships revision 20025 on their "AirStation HighPower N450 Giga".

The workaround:

Set Server mode to "Bridge (TAP)" and set the netmask, and save the settings. This will set the VPN netmask value. Then set Server mode to "Router (TUN)" and apply settings. The netmask value set in bridge mode is retained for the router mode.

Better line wrapping in Vim, 6th iteration

Here is the breakindent patch updated for Vim 7.3.682 from Fedora 17: vim-7.3.682-breakindent.patch

Migrating contacts from Android 1.x to Android 2.x

I'm finally getting around to upgrading my trusty old Android Dev Phone 1 from the original Android 1.5 firmware to Cyanogenmod 6.1. In doing so, I wanted to take my contacts with me. The contacts application changed its database schema from Android 1.x to Android 2.x, so I need to export/import. Android 2.x's contact application supports importing from VCard (.vcf) files. But Android 1.5's contact application doesn't have an export function.

So I wrote a minimal export tool.

The Android 1.x contacts database is saved in /data/com.android.providers.contacts/databases/contacts.db which is a standard sqlite3 database. I wanted contact names and phone numbers and notes, but didn't care about any of the other fields. My export tool generates a minimalistic version of .vcf that the new contacts application understands.

Example usage:

./contacts.py contacts.db > contacts.vcf
adb push contacts.vcf /sdcard/contacts.vcf

Then in the contacts application import from that file.

If you happen to have a need to export your contacts from an Android 1.x phone, this tool should give you a starting point. Note that the clean_data function fixes up some issues I had in my particular contact list, and might not be very applicable to a different data set. I'm not sure the labels ("Home", "Mobile", "Work", etc.) for the phone numbers are quite right, but then, they were already a mess in my original data. Since this was a one-off task, the code wasn't written for maintainability, and it'll probably do something awful to your data--use it at your own risk.

Building a standalone Subversion binary

There are times that you want to be able to experiment with different versions of Subversion. One scenario I've run into a number of times now is wanting to use the new patch feature of Subversion 1.7 on an "enterprise" distro or an older distro. But I don't want to upgrade everything; I just want to use it for a specific task and return to the distro-provided Subversion 1.6 for instance.

Building a standalone binary is pretty straight-forward -- enough so that it would not seem worthy of a blog post. However, I recently found myself spending an embarrassingly long time beating my head against Subversion to get a binary of the form I wanted. And the particularly galling thing about it was that I had successfully done what I was trying to replicate a mere year earlier. So, in the interest of saving others the frustration, and my self a third round of frustration, here are the steps to build a stand-alone Subversion binary:

First, you do need to be sure you have some libraries and headers available. For Fedora, you can run:

yum install apr-devel apr-util-devel neon-devel

Edited to add: If you're building Subversion >=1.8, you will also need to add sqlite-devel libserf-devel to that list.

Other distributions should be similar. I'm sure there are other development packages required, but I must have installed them at some point in the past.

Once you have your dependencies ready, go download the Subversion sourcecode. With your freshly downloaded tarball:

$ version=1.7.6
$ tar -xjf subversion-$version.tar.bz2
$ cd subversion-$version
$ ./configure --disable-shared
$ make svn
$ mv subversion/svn/svn ../svn-$version

This yields a binary named svn-1.7.6 that you can move to your ~/bin or where ever, and you can then use that specific version of Subversion when you need it. The binary will be somewhere around 8MB, give or take. This is a standalone binary, but not a completely statically linked binary; it uses the shared libraries of the system, but has all the Subversion code statically linked to the binary.

This process also works for version=1.6.18, and presumably other versions as well.

One of the interesting new toys in 1.7 is svnrdump. You can build that in essentially the same way, just with make svnrdump or make svn svnrdump instead of make svn. You'll find the binary in subversion/svnrdump/svnrdump.

Now, go, experiment with svn patch and svnrdump and all the other 1.7 goodies!

On Variable-length Integer Encoding

Suppose you want to represent data in a serialized form with the length prepended to the data. You can do something like what Pascal does with strings, and prefix it with an 8-bit length. But that only gives you 0 through 255 bytes of data. So you need something larger, such as a 64-bit value. But then a single-byte data value takes up 9 bytes including the length indicator. We'd really want small data values to use a small amount of overhead to encode the length, and we'd want the large data values to be representable, too. And thus, we want a variable-length encoding of the length. And the length is an integer, so we want a variable-length encoding of an integer. We'll start with representing a non-negative value, but a variable-length encoding of a signed integer may be worth a look too.

You can find some interesting articles on wikipedia about universal codes which are variable-length encodings of integers, but they focus on representations of integers in bit-streams. Given our usecase, we're really dealing with byte-streams.

So let's start with a simple idea: Count the number of leading 1 bits and call that N. The total size of the numeric representation is 2N bytes. Take those bytes, mask off the N leading 1 bits, and interpret the number as a binary integer.

Let's try that:

0b00000000          = 0
0b00000001          = 1
                    :
0b01111111          = 127
0b10000000 00000000 = 0
0b10000000 00000001 = 1
                    :
0b10111111 11111111 = 16383

That gives us a way to represent any non-negative integer value. But there is one undesirable characteristic of this approach: there are multiple correct ways to represent any given number. For instance, the number 0 can be represented in a single byte as 0b00000000 or in two bytes as 0b10000000 00000000. This introduces ambiguity when encoding the value 0. There may be situations where this is a desirable property, but in this case, I want there to be one and only one representation of each integer.

A simple solution is to make the representations not overlap by adding the number of valid shorter representations to the integer representation. That is, interpret the 2-byte value as an integer, then add the number of valid 1-byte values to it, And for the 4-byte value, add the number of valid 2-byte and 1-byte values to it. An alternative way to state this is to add the largest number you can represent in the 2(N-1)-byte representation (plus one) to the integer.

That gives us:

0b00000000          = 0
0b00000001          = 1
                    :
0b01111111          = 127
0b10000000 00000000 = 128
0b10000000 00000001 = 129
                    :
0b10111111 11111111 = 16511
0b11000000 00000000 00000000 00000000 = 16512
0b11000000 00000000 00000000 00000001 = 16513
                                      :
0b11011111 11111111 11111111 11111111 = 536887423

Here is a simplistic Python implementation. One of the nice things about using Python is that it can natively handle huge integers, so only the serialization aspect is needed.

This approach can be generalized in a couple of ways.

The first is that this could be done using leading 0 bits instead of leading 1 bits. I prefer the leading 1 bits because the 1-byte values 0-127 are the same as your normal unsigned char. But whether it is defined as the number of leading 1-bits or 0-bits, it still gives us a way to determine the value of N.

The second is in the translation of N into a representation size in bytes. I chose 2N, but it could just as easily be any function of N. If you wanted to have the size of the representation grow more slowly, you could use f(N) = N + 1. I like f(N) = 2N in part because it gives 1-byte, 2-byte, 4-byte, 8-byte representations that fit well into the natural integer sizes on modern computers.

This can also be generalized to signed integers as long as you define a mapping from the set of non-negative integers to the set of integers. A trivial solution would be to take the least significant bit to be a sign bit, though this gives you a way to represent negative zero. I suppose you could use that as a representation of Not-a-Number (NaN) or something along those lines. Alternatively, use a two's complement representation, though care would have to be taken with sign-extending the value and adding to that the largest magnitude negative or positive value that would overflow the next-smaller representation. This is left as an exercise to the reader.

Returning to our original problem statement, we now have a way to prepend a length to a data value while having the overhead cost low for small values while still supporting very large values. One byte of overhead to represent the length for data of 0 through 127 bytes is acceptable. Two bytes for 128 through 16511 bytes is also fine. By the time the overhead reaches 8 bytes, you're dealing with half a gigabyte of data.

But such a representation has additional possible uses. One that I have toyed with is using such a representation for a binary network communication protocol. Each message you define gets assigned an integer value, and you don't have to commit to a specific maximum number of message types when you define your protocol. Were I to use this for a protocol, I would want to make a 'version check' message have a numeric value < 128 so it fits in a single byte. And most messages would get a number that would map to a 2-byte value. That way, as messages are determined to be bandwidth "hot spots", they can be moved to a <128 value to cut a byte off their representation. The other thing I would probably do with protocol numbers would be to define a different f(N) that would grow the size of the integer representation more slowly. For that matter, it would be possible to map f(0) -> 1, f(1)->2, f(2)->2, f(3)->2, f(4)->3, etc; this would complicate some of the math, but would allow packing more values into 2 bytes. (The number of values represented by the second and third 2-byte representations would be half or a quarter of what the first 2-byte representation supported.) In a case like this, I would probably only define f(N) for the values of N I actually expect to use, and extend the definition as need arose.

Network protocols is another case where the unique nature of the representation is important. When you are dealing with systems that you want to secure (such as a network protocol), you do not want the ambiguity in the encoding process that a non-unique encoding implies. You want one and only one representation of each possible value so any attacker has no flexibility in doing something strange like using 256 bytes to represent the number 0.

I was prompted to post this by a question on programmers.stackexchange.com.