What are Blacklists and How Do They Work? (Part 1)

Originally Posted on Generator Labs Blacklist Check Service

Real-Time Blacklists (RBL) are a simple but effective way for organizations around the world, to share the location (in this case, the IP addresses) of email systems that are reputed to send email SPAM.

The most common implementation of these lists are distributed via DNS, and referred to as DNS-based Blacklists (DNSBL). Distributing this data via DNS makes perfect sense; it’s a technology that already exists, and servers all already have access to. It’s fast, and the data (IP addresses and domains) is well suited for DNS.

Types of RBLs

There are hundreds of RBLs available worldwide, managed by hundreds of organizations and individuals. Most RBLs are free to use, some are pay-to-use, and they all have their own methodologies for compiling their databases, and their own processes for being delisted.

Two of the most common and reliable methodologies for collecting data for RBLs, are based on user input, or something they call a “honey pot”.

Crowd Sourced

RBLs based on user input are the most common, and often most reliable. This data is effectively crowd sourced. When an email recipient receives a SPAM message (assuming their system supports it), they can “flag this message as SPAM”. If enough recipients flag the same message as SPAM, the IP address of the sender will be added to the RBL database.

This is common in free email services like Hotmail and Gmail.

Honey Pots

Another common source for RBL data is something called a honey pot. These are basically email addresses that are never used for any legitimate email purposes, and aren’t owned by any end users. They simply exist out on the Internet in places where robots and SPAM aggregators collect email addresses.

Any email that comes to these addresses is by definition unsolicited, and considered SPAM, and will land you on an RBL.

URIBLs

Another type of RBL is the URI Blacklist (URIBL). This is used for distributing domain names of websites that are reputed to send SPAM or to be involved in phishing schemes. So this doesn’t just affect organizations that run their own mail servers, but anyone that has a website as well.

There are definitely RBLs that are more reputable than others; there are some that have no process for being delisted, and others where you have to pay to be delisted. In my opinion, the pay-to-delist RBLs should not be considered reputable, and should not be used by mail system administrators.

Stay tuned for Part 2 of this series, where I’ll talk about how these RBLs are used by organizations, and give you some real-world examples of how RBLs can help you combat SPAM.

Generator Labs: Now with DNSSEC and TLSA

The Generator Labs domain is now signed with DNSSEC, and we’ve published the website certificate fingerprint via a TLSA record (also known as DANE), so that you can be 100% sure you’re interacting with the Generator Labs servers. 

This can be validated through the Verisign Labs Test Tool.

What is DNSSEC, and why do I care?

The point of DNSSEC is to provide a way for DNS records to be trusted by whoever receives them. The key innovation of DNSSEC is the use of public key cryptography to ensure that DNS records are authentic.

DNSSEC not only allows a DNS server to prove the authenticity of the records it returns. It also allows the assertion of “non-existence of records”.

What about DANE, what’s that all about?

DANE, or “DNS-based Authentication of Named Entities”, is a way for domain and website owners to publish SSL certificate fingerprints, so that visitors can validate that the certificate being used on a website is valid for that site.

Over the last few years, there have been several security breaches with Certificate Signing Authorities (CA’s)- companies that sign certificates that you use for your secure website- allowing the issuance of certificates for domains, not owned by the domain owners.

Using DANE, you can define exactly which certificate or CA is valid for your website, restricting the ability for a would-be hacker to masquerade as your website, by gaining access to your CA.

This obviously only makes sense in tandem with DNSSEC, as you need to validate that the DNS information providing the certificate fingerprint is valid, and not modified along the way.

How can I tell if a site is using DNSSEC/DANE?

Until there is more mainstream support for DNSSEC/DANE- for example, building support for it directly in the OS layer and in web browsers- there is a great browser plugin by cz.nic, that will show you when a site is protected with DNSSEC and DANE.

Generator Labs: Two-factor Authentication using Clef

You can now log in to your Generator Labs account using secure, two-factor authentication using Clef.

Clef is secure two-factor authentication using your mobile device, with no passwords or tokens, and no personal data is shared between Generator Labs and Clef.

Simply download the Clef Mobile App, and then register your Generator Labs account with Clef from the Profile page of your Generator Labs account.

Once registered, you can log in and log out of the Generator Labs interface using your mobile device- it’s that easy! If you have any questions or comments about using Clef, please let us know @ info@generatorlabs.com

Two-factor authentication is available to all Generator Labs accounts. Sign up Today!

Net_DNS2 Moved to GitHub

I’ve never been a bit fan of git- I’ve got used to using SVN over the years, and never saw a compelling reason to change- until now- that Google is shutting down the Google Code service- so I’m forced to move.

Luckily I can still keep using SVN with GitHub- I can put off actually using git for the foreseeable future!

The new Net_DNS2 repository is officially moved to GitHub:

https://github.com/mikepultz/netdns2

DNS in PHP: How to Use the Net_DNS2 Library

The PEAR Net_DNS2 DNS resolver library has been around for a while now- I originally wrote it in late 2010, with the latest release just a few months ago.

Net_DNS2, much like its predecessor Net_DNS, is a native DNS resolver/updater- which means it does not use system commands and is not a language binding on top of a C library, but instead, uses UDP/TCP sockets to communicate directly with DNS servers to retrieve the requested information.

Net_DNS2 will use the name servers specified in your resolv.conf file (for *nix users), or you can specify which name severs to use directly in the config.

Simple Lookup Example

This example uses the Google public DNS servers, to look up the A records for google.com:

$r = new Net_DNS2_Resolver(array('nameservers' => array('8.8.8.8')));    

try
{
    $result = $r->query('google.com', 'A');     

    foreach($result->answer as $record)
    {
        echo $record->address, "\n";
    }

} catch(Net_DNS2_Exception $e)  
{
    echo "::query() failed: ", $e->getMessage(), "\n";    
}

The result is:

66.185.85.45
66.185.85.34
66.185.85.59
66.185.85.30
66.185.85.54
66.185.85.35
66.185.85.39
66.185.85.55
66.185.85.49
66.185.85.25
66.185.85.40
66.185.85.24
66.185.85.44
66.185.85.29
66.185.85.20
66.185.85.50

Net_DNS2 currently supports 58 different resource record types, including all the resource records required for DNSSEC, and some resource records that have only been defined a few months ago, like the OPENPGPKEY record.

Here is an example looking up the MX records (for mail delivery) for gmail.com:

$r = new Net_DNS2_Resolver(array('nameservers' => array('8.8.8.8')));
try
{
    $result = $r->query('gmail.com', 'MX');

    foreach($result->answer as $record)
    {
        printf("preference=%2d, host=%s\n", $record->preference, $record->exchange);
    }

} catch(Net_DNS2_Exception $e)
{
    echo "::query() failed: ", $e->getMessage(), "\n";
}

The result is:

preference=40, host=alt4.gmail-smtp-in.l.google.com
preference=20, host=alt2.gmail-smtp-in.l.google.com
preference= 5, host=gmail-smtp-in.l.google.com
preference=30, host=alt3.gmail-smtp-in.l.google.com
preference=10, host=alt1.gmail-smtp-in.l.google.com

Simple Update Example

Net_DNS2 can also be used to make dynamic DNS updates. This example updates the MX record for the domain “example.com”. For updates, the DNS server you want to specify is the authoritative DNS server for the domain, and not simply a resolver:

$u = new Net_DNS2_Updater('example.com', array('nameservers' => array('192.168.0.1')));

try  
{
    //    
    // create a new MX RR object to add to the example.com zone    
    //    
    $mx = Net_DNS2_RR::fromString('example.com MX 10 mail.google.com');         

    //    
    // add the record    
    //    
    $u->add($mx);    

    //    
    // add a TSIG RR to authenticate the request 
    //    
    $u->signTSIG('my-key', '9dnf93asdf39fs');    

    //    
    // execute the request    
    //    
    $u->update();    

} catch(Net_DNS2_Exception $e)  
{
        echo "::update() failed: ", $e->getMessage(), "\n";
}

Net_DNS2 supports authentication via TSIG or SIG(0) (current supports RSA keys only); this is often required for sending DNS updates (as in the example above), or for making full zone-transfer requests, like this:

$r = new Net_DNS2_Resolver(array('nameservers' => array('8.8.8.8')));

//
// sign with TSIG to authenticate the zone transfer
//
$r->signTSIG('mykey', '9dnf93asdf39fs');

try
{
    $result = $r->query('example.com', 'AXFR');

    foreach($result->answer as $record)
    {
        echo $record;
    }

} catch(Net_DNS2_Exception $e)
{
    echo "::query() failed: ", $e->getMessage(), "\n";
}

Net_DNS2 is available as a PEAR module, or via Packagist; you can also find out more on the Net_DNS2 website.