Archive

Archive for November, 2010

Decrypt HTTPS Traffic Using Wireshark And Key File

November 16th, 2010 No comments

wireshark-logoWireshark is a useful tool in troubleshooting. Wireshark can decrypt SSL traffic as long as you have the server private key. This can be extremely useful, if you have to debug HTTPS traffic and cannot use HTTP instead.

First we will capture a HTTPS traffic for our testing. Here our HTTPS server’s ip address is 192.168.x.x and the port is default 443. I prefer to use tcpdump for packet capture but you can do it using the Wireshark.

The below command will capture all the encrypted traffic to and from from our server.

$ sudo tcpdump -w /tmp/ssl.pcap  -ni eth0 -s0  host 192.168.x.x port 443

The captured data will go to the ssl.pcap file. Once you have the captured packets in the file open it in the Wireshark. Use the “Follow TCP Stream” options and you can see the encrypted data.

Screenshot-Follow TCP Stream
Next thing we need is the server’s private key. Once you have the key file to decrypt the traffic, just goto “Edit -> Preferences”. Now on the left side menu choose “Protocols -> SSL”. Fill “RSA Key list” field in the format <host>, <port>, <protocol>, <key_file>. ie We will specify the server’s IP address, the port on which the server listens and the path to the server’s private key. The file format needed for the server’s private key is PEM. In our example it is 192.168.x.x, 443, https, /path/to/keyfile.pem.

wireshark
Now Apply the setting and return to main window.

Now if you click on each row you can see a “Decrypted SSL Data (size) “ tab on the bottom of “Packet Bytes” frame. This tab will be shown if there is any decrypted data available.

Screenshot-ssl.pcap - Wireshark-1

You can now use the “Follow SSL Stream” option to view the decrypted data stream.

Screenshot-ssl.pcap - Wireshark

Happy decrypting ;-)

Categories: HOW-TOS Tags: , , , ,

[Java-Tip] Non-Blocking Method To Download Files From Web

November 9th, 2010 No comments

java-150x150
The URLConnection class contains many methods that let you communicate with the URL over the network. But the URLConnection doesn’t provide a callback mechanism to know the data read progress. Java’s support of interfaces provides a mechanism by which we can get the equivalent of callbacks. The trick is to define a simple interface that declares the method we wish to be invoked and to notify the data read progress of a URLConnection.

We define our Event as follows.

// FileDownloadEvent.java
 
public interface FileDownloadEvent
{
    // This is just a regular method so it can return something or
    // take arguments if you like.
    public void dataReadProgress (int done, int total, byte data[]);
    public void done(boolean error);
}

There is two methods, the dataReadProgress() and thedone() method. We invoke the dataReadProgress() each time we read a chunk of data to notify the data read progress. We use the done() method to inform the data read is over or an error has happened.

The class that will signal the event needs to expect objects that implement the dataReadProgress interface and then invoke the dataReadProgress() method as appropriate.
We will keep a counter for the downloaded data and fire the dataReadProgress event each time we read a chunk of data.

// FileDownload.java
 
import java.net.URL;
import java.net.URLConnection;
import java.io.InputStream;
import java.io.DataInputStream;
import java.io.BufferedInputStream;
import java.util.Arrays;
import java.lang.Thread;
import java.lang.Runnable;
 
public class FileDownload extends Thread implements Runnable
{
    private FileDownloadEvent ie;
    private InputStream is = null;
    private DataInputStream dis = null;
    private int dataReadSize = 4096;
    private String downloadURL = null;
 
    public FileDownload (FileDownloadEvent event)
    {
        // Save the event object for later use.
        ie = event;
    }
 
    public void request (String url)
    {
        this.downloadURL = url;
	this.start();
    }
 
    //...
    public void run ()
    {
        boolean error = false;
        try {
            URL url = new URL(this.downloadURL);
	    URLConnection fdCon = url.openConnection();
 
            int total = fdCon.getContentLength();
 
            is = url.openStream();  // throws an IOException
            dis = new DataInputStream(new BufferedInputStream(is));
 
	    byte[] data = new byte[dataReadSize];
	    int progress = 0, n;
            while ((n = dis.read(data)) &gt; 0) {
	        progress += n;
                this.ie.dataReadProgress (progress, total, data);
                Arrays.fill(data, (byte)0);
            }
        } catch (Exception e)
        {
            error = true;
        }
        this.ie.done(error);
    }
    // ...
}

The code that wishes to receive the event notification must implement the FileDownloadEvent interface and just pass a reference to itself to the event notifier or do as in the below code.

// Download.java
 
public class Download
{
    public static void main(String args[])
    {
        // Create the event notifier and pass ourself to it.
        FileDownload req = new FileDownload (new FileDownloadEvent() {
            // Define the actual handler for the event.
            public void dataReadProgress (int done, int total, byte[] data)
            {
                System.out.println("Progress: " + ((float)done/(float)total) * 100 + "%");
                // Do something with data...
            }
            public void done (boolean error)
            {
		System.out.println("Download Completed.");
                // Do something...
            }
        });
 
        req.request("http://somedomain/path/to/file.gz");
 
	// Do something
    }
}

That’s all there is to it. I hope use this simple Java idiom will be useful to someone.

Categories: JAVA Tags: , , ,

Playing With Python And CouchDB

November 4th, 2010 No comments

couchdb
Apache CouchDB is a document-oriented database that can be queried and indexed in a MapReduce fashion using JavaScript. CouchDB also offers incremental replication with bi-directional conflict detection and resolution.

CouchDB provides a RESTful JSON API than can be accessed from any environment that allows HTTP requests. There are myriad third-party client libraries that make this even easier from your programming language of choice. CouchDB’s built in Web administration console speaks directly to the database using HTTP requests issued from your browser.

CouchDB is written in Erlang, a robust functional programming language ideal for building concurrent distributed systems. Erlang allows for a flexible design that is easily scalable and readily extensible.

See the introduction and the technical overview for more information.

Getting Started

The couchdb.client.Server represents a CouchDB server. New databases can be created using the create method:

from couchdb.client import Server
 
server = Server('http://localhost:5984')
print server
try:
        db = server.create('emails')
except Exception:
        db = server['emails']
 
print db

Output:

<database 'emails'>

This class behaves like a dictionary of databases. For example, to get a list of database names on the server, you can simply iterate over the server object.

for db in server:
    print db

The out put will be something like

test
segfault
emails

Creating a Document

To define a document mapping, you declare a Python class inherited from Document, and add any number of Field attributes:

from couchdb.client import Server, Document
from couchdb.mapping import TextField, DateTimeField, ListField

Now you create subclass of Document and fill in the values.

class Email(Document):
  frm  = TextField()
  to = ListField(TextField())
  sub = TextField()
  added = DateTimeField(default=datetime.now())
 
eml = Email()
eml['frm'] = "vinod@example.com"
eml['to'] = "user@domain.com"
eml['sub'] = "test"

To update a document, simply set the attributes, and then call the save() method:

doc_id, doc_rev = db.save(eml)
print doc_id, doc_rev

Output:

8be7ef2e5d711f11e859972ca9d38a52 455397369

Retrieving documents

for docid in db:
  eml = db.get(docid)
  print eml['frm'], eml['to'], eml['sub']

Output:

vinod@example.com user@domain.com test

Working With Views

Views are the primary tool used for querying and reporting on CouchDB documents. There are two different kinds of views: permanent and temporary views.

Temporary Views

The views you don’t want to save in the CouchDB database. NOTE: Temporary views are only good during development. Final code should not rely on them as they are very expensive to compute each time they get called and they get increasingly slower the more data you have in a database.

code = '''function(doc) { if(doc.frm == "vinod@example.com") emit(doc.frm, null); }'''
results = db.query(code)
 
for res in results:
   print res.key
Permenant Views

Permanent views are stored inside special documents called design documents, and can be accessed via an HTTP GET request to the URI /{dbname}/{docid}/{viewname}, where {docid} has the prefix _design/ so that CouchDB recognizes the document as a design document, and {viewname} has the prefix _view/ so that CouchDB recognizes it as a view.

You use ViewDefinition class to create a permanent view in the database.

from couchdb.design import ViewDefinition
 
rpt_view = ViewDefinition('reports', 'fromemail', '''function(doc) { if(doc.frm == "vinod@example.com") emit(doc.frm, null); }''')
rpt_view.sync(db)

You can see our view in the drop down list.
couchdb

Query permanent views
for res in db.view("_design/reports/_view/fromemail"):
        print res.id, res.key

Output

8be7ef2e5d711f11e859972ca9d38a52 vinod@example.com

Installing Python-CouchDB

If you are a Debian/Ubuntu user install couchdb and python-couchdb via apt:

sudo aptitude install couchdb python-couchdb

Or you can download from python-couchdb project home.

Categories: PYTHON Tags: , ,