segfault.in

This is the third post in the article series “Playing With Python And Gmail”. This will be a tutorial on how to send mails using Python smtplib through Gmail SMTP.

The smtplib module defines an SMTP client session object that can be used to send mail to any Internet machine with an SMTP or ESMTP listener daemon.

[vinod@mercury ~]$ python
Python 2.5.2 (r252:60911, Jan 24 2010, 14:53:14)
[GCC 4.3.2] on linux2
Type "help", "copyright", "credits" or "license" for more information.
>>> import smtplib

The first step is to create a SMTP connection to the server. The smtplib.SMTP class encapsulates an SMTP connection. It has methods that support a full repertoire of SMTP and ESMTP operations. If the optional host and port parameters are given, the SMTP connect() method is called with those parameters during initialization. An SMTPConnectError is raised if the specified host doesn’t respond correctly. The optional timeout parameter specifies a timeout in seconds for blocking operations like the connection attempt (if not specified, the global default timeout setting will be used).

Remember Google’s SMTP server is ‘smtp.gmail.com’ and the port is 587.

>>> session = smtplib.SMTP('smtp.gmail.com', 587)

Next we will identify ourself to an ESMTP server using EHLO. The SMTP HELO clause is the stage of the SMTP protocol where a SMTP server introduce them selves to each other. EHLO is just like HELO except that the server’s response text provides computer-readable information about the server’s abilities.

>>> session.ehlo()
(250, 'mx.google.com at your service, [x.x.x.x]\nSIZE 35651584\n8BITMIME\nSTARTTLS\nENHANCEDSTATUSCODES')

Next we call SMTP.starttls function to put the SMTP connection in TLS (Transport Layer Security) mode. All SMTP commands that follow will be encrypted. You should then call ehlo() again.

If keyfile and certfile are provided, these are passed to the socket module’s ssl() function.If there has been no previous EHLO or HELO command this session, this method tries ESMTP EHLO first.

>>> session.starttls()
(220, '2.0.0 Ready to start TLS')
>>> session.ehlo
>
>>>

OK, now we are safe to login to the server using SMTP.login(user, password). After successful login we use SMTP.sendmail(from_addr, to_addrs, msg[, mail_options, rcpt_options]) to send mails via Gmail.

>>> session.login('username@gmail.com', 'topsecret')
(235, '2.7.0 Accepted')

We will cerate some SMTP headers before sending our mail.

headers = ["from: " + sender,
           "subject: " + subject,
           "to: " + recipient,
           "mime-version: 1.0",
           "content-type: text/html"]
headers = "\r\n".join(headers)

Finally send our mail

session.sendmail(sender, recipient, headers + "\r\n\r\n" + body)

Sample code

#!/usr/bin/python
import smtplib
 
SMTP_SERVER = 'smtp.gmail.com'
SMTP_PORT = 587
 
sender = 'you@gmail.com'
recipient = 'user@example.com'
subject = 'Gmail SMTP Test'
body = 'blah blah blah'
 
"Sends an e-mail to the specified recipient."
 
body = "" + body + ""
 
headers = ["From: " + sender,
           "Subject: " + subject,
           "To: " + recipient,
           "MIME-Version: 1.0",
           "Content-Type: text/html"]
headers = "\r\n".join(headers)
 
session = smtplib.SMTP(server, port)
 
session.ehlo()
session.starttls()
session.ehlo
session.login(sender, password)
 
session.sendmail(sender, recipient, headers + "\r\n\r\n" + body)
session.quit()

Sending Attachments

The email package is a library for managing email messages, including MIME and other RFC 2822-based message documents. The central class in the email package is the Message class, imported from the email.message module. It is the base class for the email object model. Message provides the core functionality for setting and querying header fields, and for accessing message bodies. This is the base class for all the MIME-specific subclasses of Message. Ordinarily you won’t create instances specifically of MIMEBase, although you could. MIMEBase is provided primarily as a convenient base class for more specific MIME-aware subclasses.

We import MIMEMultipart class, a sub class of MIMEBase to create the enclosing (outer) message.

from email.MIMEMultipart import MIMEMultipart
 
msg = MIMEMultipart()
msg['From'] = sender
msg['To'] = recipient
msg['Subject'] = subject

The MIMEImage class is used to create MIME message objects of major image types. We create a MIMEImage object containing out raw image data and attach it to our outer message.

from email.MIMEImage import MIMEImage
 
img = MIMEImage(open("test.gif","rb").read(), _subtype="gif")
 
img.add_header('Content-Disposition', 'attachment; filename="test.gif"')
msg.attach(img)

Finally we create MIMEText object to add some plain text to our mail.The MIMEText class is used to create MIME objects of type text.

from email.MIMEText import MIMEText
 
part = MIMEText('text', "plain")
part.set_payload("Image attached.")
msg.attach(part)

Now send the mail

session.sendmail(sender, recipient, msg.as_string())

You can create MIME objects for audio, application etc. Find the documentation here.

A Simple Example

Send all .gif files in a directory as attachment.

#!/usr/bin/python
import os, re
import sys
import smtplib
 
from email.mime.image import MIMEImage
from email.mime.multipart import MIMEMultipart
from email.mime.text import MIMEText
 
SMTP_SERVER = 'smtp.gmail.com'
SMTP_PORT = 587
 
sender = 'you@gmail.com'
password = "secret"
recipient = 'user@example.com'
subject = 'Python emaillib Test'
message = 'Images attached.'
 
directory = "/tmp/images/"
 
def main():
    msg = MIMEMultipart()
    msg['Subject'] = 'Python emaillib Test'
    msg['To'] = recipient
    msg['From'] = sender
 
    files = os.listdir(directory)
    gifsearch = re.compile(".gif", re.IGNORECASE)
    files = filter(gifsearch.search, files)
    for filename in files:
        path = os.path.join(directory, filename)
        if not os.path.isfile(path):
            continue
 
        img = MIMEImage(open(path, 'rb').read(), _subtype="gif")
        img.add_header('Content-Disposition', 'attachment', filename=filename)
        msg.attach(img)
 
    part = MIMEText('text', "plain")
    part.set_payload(message)
    msg.attach(part)
 
    session = smtplib.SMTP(SMTP_SERVER, SMTP_PORT)
 
    session.ehlo()
    session.starttls()
    session.ehlo
    session.login(sender, password)
 
    session.sendmail(sender, recipient, msg.as_string())
    session.quit()
 
if __name__ == '__main__':
    main()

This is the second part of the article series ‘Playing With Python And Gmail’. If you didn’t read the first part I would recomend you to read it.

This time we will see how to fetch mails from Gmail using Python.

Reading Mails

The IMAP4.fetch method fetch (parts of) messages. message_parts should be a string of message part names enclosed within parentheses, eg: “(UID BODY[TEXT])”. Returned data are tuples of message part envelope and data.

Here is a minimal example (without error checking) that opens a mailbox and retrieves and prints all messages:

import imaplib
 
M = imaplib.IMAP4('imap.gmail.com', 993)
M.login('myname@gmail.com', 'pa$$word')
M.select()
typ, data = M.search(None, 'ALL')
for num in data[0].split():
    typ, data = M.fetch(num, '(RFC822)')
    print 'Message %s\n%s\n' % (num, data[0][1])
M.close()
M.logout()

The email package provides a standard parser that understands most email document structures, including MIME documents. You can pass the parser a string or a file object, and the parser will return to you the root Message instance of the object structure. For simple, non-MIME messages the payload of this root object will likely be a string containing the text of the message. For MIME messages, the root object will return True from its is_multipart() method, and the subparts can be accessed via the get_payload() and walk() methods.

Extract Mail Headers
Here is method to retrieve from, to and subject from from an email message:

from email.parser import HeaderParser
 
resp, data = M.FETCH(1, '(RFC822)')
msg = HeaderParser().parsestr(data[0][1])
 
print msg['From']
print msg['To']
print msg['Subject']
 
M.LOGOUT()

Output will be something like.

Gmail Team
My Name
Gmail is different. Here's what you need to know.

Identifying the content type
The Content-Type header indicates the Internet media type of the message content, consisting of a type and subtype, for example text/plain is the default value for “Content-Type:”
Gmail uses alternative content, such as a message sent in both plain text and another format such as HTML (multipart/alternative with the same content in text/plain and text/html forms).

import email
 
resp, data = M.FETCH(1, '(RFC822)')
mail = email.message_from_string(data[0][1])
 
for part in mail.walk():
  print 'Content-Type:',part.get_content_type()
  print 'Main Content:',part.get_content_maintype()
  print 'Sub Content:',part.get_content_subtype()

Out put will be

Content-Type: multipart/alternative
Main Content: multipart
Sub Content: alternative
Content-Type: text/plain
Main Content: text
Sub Content: plain
Content-Type: text/html
Main Content: text
Sub Content: html

Extract Message Body.
Using the walk() method we can iterate through Message parts. The get_payload() method will return the current payload, which will be a list of Message objects when is_multipart() is True, or a string when is_multipart() is False.

import email
 
resp, data = M.FETCH(1, '(RFC822)')
mail = email.message_from_string(data[0][1])
 
for part in mail.walk():
  # multipart are just containers, so we skip them
  if part.get_content_maintype() == 'multipart':
      continue
 
  # we are interested only in the simple text messages
  if part.get_content_subtype() != 'plain':
    continue
 
  payload = part.get_payload()
  print payload
 
M.LOGOUT()

Extracting Attachmets
The below code will extract and save attached images to disk.

import re
 
name_pat = re.compile('name=\".*\"')
 
for part in mail.walk():
  if part.get_content_maintype() != 'image':
    continue
 
  file_type = part.get_content_type().split('/')[1]
  if not file_type:
    file_type = 'jpg'
 
  filename = part.get_filename()
  if not filename:
    filename = name_pat.findall(part.get('Content-Type'))[0][6:-1]
 
  counter = 1
  if not filename:
    filename = 'img-%03d%s' % (counter, file_type)
    counter += 1
 
  payload = part.get_payload(decode=True)
 
  if not os.path.isfile(filename) :
      # finally write the stuff
      fp = open(filename, 'wb')
      fp.write(part.get_payload(decode=True))
      fp.close()

That’s it. In the next part I will explain searching and moving your mails using Python. Dont forget to subscribe 

In addition to its web interface Google also provides access via IMAP. The python imaplib module defines three classes, IMAP4, IMAP4_SSL and IMAP4_stream, which encapsulate a connection to an IMAP4 server and implement a large subset of the IMAP4rev1 client protocol as defined in RFC 2060.

The IMAP4 class implements the actual IMAP4 protocol. The connection is created and protocol version (IMAP4 or IMAP4rev1) is determined when the instance is initialized.

Getting started with Python Imaplib
To start with, we will create a simple python program to login to Gmail via IMAP.

import imaplib
 
IMAP_SERVER='imap.gmail.com'
IMAP_PORT=993
 
M = imaplib.IMAP4_SSL(IMAP_SERVER, IMAP_PORT)
rc, resp = M.login('username@gmail.com', 'pa$$word')
print rc, resp
 
M.logout()

IMAP4.IMAP4_SSL is a subclass derived from IMAP4 that connects over an SSL encrypted socket (to use this class you need a socket module that was compiled with SSL support). If host is not specified, ” (the local host) is used. If port is omitted, the standard IMAP4-over-SSL port (993) is used. keyfile and certfile are also optional – they can contain a PEM formatted private key and certificate chain file for the SSL connection.

If authentication is successful the output will be:

OK ['username@gmail.com authenticated (Success)']

As part of our exercise we will be writing may usefull functions. It is good to create a python class file to put our functions so that at the end of our exercise we will have a cool python gmail library. Lets create a pygmail.py

import imaplib
 
class pygmail:
  def __init__(self):
    self.IMAP_SERVER='imap.gmail.com'
    self.IMAP_PORT=993
    self.M = None
    self.response = None
 
  def login(self, username, password):
    self.M = imaplib.IMAP4_SSL(self.IMAP_SERVER, self.IMAP_PORT)
    rc, self.response = self.M.login(username, password)
    return rc
 
  def logout(self):
    self.M.logout()

g = pygmail()
g.login('username@gmail.com', 'pa$$word')
print g.response

Listing Mailboxes
The IMAP4.list() function list mailbox names in directory matching pattern. The directory defaults to the top-level mail folder, and pattern defaults to match anything. Returned data contains a list of LIST responses.

Add the below function to our pygmial.py

def get_mailboxes(self):
  rc, self.response = self.M.list()
  for item in self.response:
    self.mailboxes.append(item.split()[-1])
  return rc

Use:

g.get_mailboxes()
for item in g.mailboxes:
  print item

This will output your Gmail mailboxes

"INBOX"
"Sent"
"Trash"
"/"
"[Gmail]/All
"[Gmail]/Drafts"
"[Gmail]/Sent
"[Gmail]/Spam"
"[Gmail]/Starred"
"[Gmail]/Trash"
"freebsd-net"
"fsug-tvm"
"openflow"

Creating, Renaming, Deleting Mailboxes
IMAP4.create, IMAP4.rename, IMAP4.delete functions will create, rename, delete the mailboxes respectively.
Lets add three more functions to out lib.

def rename_mailbox(self, oldmailbox, newmailbox):
  rc, self.response = self.M.rename(oldmailbox, newmailbox)
  return rc
 
def create_mailbox(self, mailbox):
  rc, self.response = self.M.create(mailbox)
  return rc
 
def delete_mailbox(self, mailbox):
  rc, self.response = self.M.delete(mailbox)
  return rc

Get Mail Count
The IMAP4.select function select a mailbox. Returned data is the count of messages in mailbox (EXISTS response). The default mailbox is 'INBOX'. If the readonly flag is set, modifications to the mailbox are not allowed.

Add the get_mail_count function to out Python class.

  def get_mail_count(self, folder='Inbox'):
    rc, count = self.M.select(folder)
    return count[0]

Output:

14581

You can specify your mailbox also.

g.get_mail_count('mailbox')

Get Unread Mail Count
The IMAP4.status() function request named status conditions for mailbox. The standard defines these status conditions:

MESSAGES - The number of messages in the mailbox.
RECENT - The number of messages with the Recent flag set.
UIDNEXT - The next unique identifier value of the mailbox.
UIDVALIDITY - The unique identifier validity value of the mailbox.
UNSEEN - The number of messages which do not have the Seen flag set.

Using the UNSEEN condition will return total unread messages in Inbox.

Lets define a function to get unread mail count in our pygmail.py.

  def get_unread_count(self, folder='Inbox'):
    rc, message = self.M.status(folder, "(UNSEEN)")
    unreadCount = re.search("UNSEEN (\d+)", message[0]).group(1)
    return unreadCount

Output:

2

OK, enough for a start. In the next parts of this article I will explain sending, searching, retrieving mails from Gmail via Python. So dont forget to subscribe

I am pushing our small Python Gmail library to github hopefully useful to someone.
PyGmail: http://github.com/vinod85/pygmail