How to Download Emails with Image Attachments from Gmail with PHP

If you’ve read “AWS Scripted”, you will have noticed that I recommend disabling file uploads because they pose a major security threat. I also suggest using email with an attachment as an alternative. Whilst not as clean as a file upload, and certainly more inconvenient for your users, this method is inherently much more secure. You could also use the code below to read emails without attachments from an account.

As an alternative, you could set up a completely separate server which only deals with file uploads. This is ok but I would not recommend keeping any sensitive data on it. And if your users are uploading things like identity documents, it would be a disaster if these were compromised. So at the very least, you need to send sensitive data off-server. However, you also need to make sure that wherever you send the data is also secure and not hackable from the file upload server. What you need is a ‘one-way’ passage to send the data down. You could achieve this by saving attachments to a database, but only allowing write access to the table from the credentials on the file upload server. You would enable read access only from a secure administration server with which you process the data.

In any case, below you will find PHP code to download emails and attachments from Gmail. Here is a possible scenario: you need an identity document from your user; you ask them to send an email with scan of said document attached to an email address you control; you have set up this email address (something like docs@yourcomany.com) to forward emails to a highly secure Gmail account (see below); you periodically run this script to fetch emails and attachments from Gmail and send them to your database; you have an Admin interface which allows you to view and process received emails. This is very secure because there is no public access to this server, only Admin access, which you could control with strong passwords, AWS Security Groups or firewalls.

Emails are inherently messy. There are lots of ways attachments can be attached, you’re dealing with MIME types and assorted paraphernalia. It took me a long time to get the code below working, and a couple of times I had to rewrite because a new style of email came in. Be aware that you might get an unintelligible email format and might need to add processing code.

Gmail is a good choice for this exercise because it is very secure if you turn on 2-Step Verification and it gives you lots of free space to store emails and attachments. You can keep the Gmail emails as a record or delete the emails as you prefer, but I would advise deleting only once human has verified the contents of an email. In the script below, we move the read emails to a ‘processed’ folder.

Gmail used to allow you to send and receive email from different email addresses than your default account address. But they have now disabled this very useful feature (it meant you could use Gmail to handle inbound and outbound emails from addresses on your domain). But you can still forward emails to your Gmail account, so with your Domain Registrar set up a new email address and forward it to your Gmail Account. Note that, in their infinite wisdom, Google don’t display emails sent from the receiving email account to the forwarding one – you just get one copy in your Sent Mail folder. But if you send an email to the forwarding address from a different account, it will appear.

On your Gmail account, you should definitely enable 2-Step Verification. However, this means you will need to create an app specific password to be used in the script below. Note that this password should be kept very secure, as it bypasses 2-Step Verification, so a Secure Laptop strategy is in order (see ‘AWS Scripted’ for more information on this). To create your app specific password:

  • go to your Google Account settings (click your picture top right and select ‘Account’)
  • select the ‘Security’ tab
  • click ‘Settings’ for ‘App passwords’
  • in the ‘App passwords’ box, select ‘Mail’ from the first drop-down
  • select Other (custom name) from the ‘Select Device’ drop-down
  • type a name, such as ‘PHP’
  • click ‘Generate’
  • copy the password shown into your PHP script

Also, in Gmail, you will need to create a new Label ‘processed’. On the left menu click ‘More’, then ‘Create new label’, then enter ‘processed’ as the label name and leave ‘Nest label under’ unchecked. Then press ‘Create’.

OK, so here is the PHP script which does the downloading:

<?php

// gmail credentials
$gmail_username='<your gmail email address>';
$gmail_password='<your gmail app password>';

// can be called without signing in
$signin=1;
require "../secure.php";
	
function get_date() {
	return date_format(date_create(), 'Y-m-d H:i:s');
	}

function get_decode_value($message, $encoding) {
	switch($encoding) {
		case 0:case 1:$message = imap_8bit($message); break;
		case 2:$message = imap_binary($message); break;
		case 3:case 5:$message=imap_base64($message); break;
		case 4:$message = imap_qprint($message); break;
		}
	return $message;
	}

function connectgmail($nusername, $npassword) {
	// connect to gmail
	$hostname = '{imap.gmail.com:993/imap/ssl}';
	// try to connect
	$inbox = imap_open($hostname, $nusername, $npassword) or die('Cannot connect to Gmail: ' . imap_last_error());
	return $inbox;
	}

function getattachment($ninbox, $nemail_number, $nid) {
	$message = array();
	$message["attachment"]["type"][0] = "text";
	$message["attachment"]["type"][1] = "multipart";
	$message["attachment"]["type"][2] = "message";
	$message["attachment"]["type"][3] = "application";
	$message["attachment"]["type"][4] = "audio";
	$message["attachment"]["type"][5] = "image";
	$message["attachment"]["type"][6] = "video";
	$message["attachment"]["type"][7] = "other";
	$structure = imap_fetchstructure($ninbox, $nemail_number);
	if (!isset($structure->parts))
		return "";
	$parts = $structure->parts;
	$message["pid"][1] = (1);
	if (count($parts)<=1)
		return "";
	$part = $parts[1];
	$message["type"][1] = $message["attachment"]["type"][$part->type] . "/" . strtolower($part->subtype);
	$message["subtype"][1] = strtolower($part->subtype);
	if (!isset($part->dparameters[0]->value))
		return "";
	$filename = $part->dparameters[0]->value;
	$body = imap_fetchbody($ninbox, $nemail_number, 2);
	$filename=$nid.".".substr($filename, strrpos($filename, '.')+1);
	$dst = "../attachments/".$filename;
	$fp = fopen($dst, 'w');
	$data = get_decode_value($body, $part->type);
	fputs($fp, $data);
	fclose($fp);
	return $filename;
	}

function getinlineattachment($ninbox, $nemail_number, $nid) {
	$body = imap_fetchbody($ninbox, $nemail_number, 1);
	$imgdata=base64_decode($body);
	$imgtype=getImageMimeType($imgdata);
	if ($imgtype=="")
		return "";
	$filename=$nid.".".$imgtype;
	$dst = "../attachments/".$filename;
	$fp = fopen($dst, 'w');
	fputs($fp, $imgdata);
	fclose($fp);
	return $filename;
	}

function getinlineattachment2($ninbox, $nemail_number, $nid) {
	$body = imap_fetchbody($ninbox, $nemail_number, 2);
	$dr=strpos($body, "Content-Transfer-Encoding: base64");
	$body=substr($body, $dr);
	$dr=strpos($body, "\r\n\r\n");
	$body=substr($body, $dr);
	$imgdata=base64_decode($body);
	$imgtype=getImageMimeType($imgdata);
	if ($imgtype=="")
		return "";
	$filename=$nid.".".$imgtype;
	$dst = "../attachments/".$filename;
	$fp = fopen($dst, 'w');
	fputs($fp, $imgdata);
	fclose($fp);
	return $filename;
	}

function getBytesFromHexString($hexdata) {
	for($count = 0; $count < strlen($hexdata); $count+=2)
		$bytes[] = chr(hexdec(substr($hexdata, $count, 2)));
	return implode($bytes);
	}

function getImageMimeType($imagedata) {
	$imagemimetypes = array("jpg" => "FFD8", "png" => "89504E470D0A1A0A", "gif" => "474946", "bmp" => "424D", "tif" => "4949", "tif" => "4D4D");
	foreach ($imagemimetypes as $mime => $hexbytes) {
		$bytes = getBytesFromHexString($hexbytes);
		if (substr($imagedata, 0, strlen($bytes)) == $bytes)
			return $mime;
		}
	return NULL;
	}

// set your gmail credentials here: email address and app password
$inbox=connectgmail($gmail_username, $gmail_password);

$emails = imap_search($inbox,'ALL');
if($emails) {
	sort($emails);
	foreach($emails as $email_number) {
		// get details
		$overview = imap_headerinfo($inbox, $email_number);
		$emailfrom=$overview->fromaddress;
		$emailsubject=$overview->subject;
		$emaildate=$overview->date;
		$emailmessage=imap_fetchbody($inbox, $email_number, 1.1);
		// message to be outputted
		echo get_date()." found email ID: ".$email_number." from: ".htmlentities($emailfrom)." subject: ".$emailsubject." message: ".$emailmessage."<br>";
		// clean data
		$emailfrom=substr($emailfrom, 0, 255);
		$emailsubject=substr($emailsubject, 0, 255);
		// insert into db and get id
		$result=doSQL("INSERT INTO receivedemails (emailfrom, emailsubject, emaildate, emailmessage) VALUES (?, ?, ?, ?);", $emailfrom, $emailsubject, $emaildate, $emailmessage) or die("Error inserting to receivedemails<br>".mysqli_error($db));
		$newid=$db->insert_id;
		// try to download the image as an attachment
		$hasattachment=1;
		$filename="";
		try {
			$filename=getattachment($inbox, $email_number, $newid);
			if ($filename=="")
				$hasattachment=0;
			}
		catch (Exception $e) {
			$hasattachment=0;
			}
		if ($hasattachment==0) {
			// try to download the image as an inline 1
			$filename=getinlineattachment($inbox, $email_number, $newid);
			if ($filename=="")
				$hasattachment=0;
			else
				$hasattachment=1;
			}
		if ($hasattachment==0) {
			// try to download the image as an inline 2
			$filename=getinlineattachment2($inbox, $email_number, $newid);
			if ($filename=="")
				$hasattachment=0;
			else
				$hasattachment=1;
			}
		$result=doSQL("update receivedemails set hasattachment=?, filename=? where receivedemailID=?;", $hasattachment, $filename, $newid) or die("Error updating receivedemails\n".mysqli_error($db));
		echo "attachment: ".(($hasattachment==1)?"yes":"no")."<br>";
		}

	// move to processed
	$elist="";
	foreach($emails as $email_number)
		$elist=$elist.$email_number.",";
	$elist=substr($elist, 0, -1);
	imap_mail_move($inbox, $elist, "processed");
	echo get_date()." email IDs: ".$elist." moved to processed<br>";
	}

imap_close($inbox);

?>

done

You’ll also want to be able to view received emails from the database, so I have provided a simple, configurable viewing script. The following zip contains a skeleton Admin website with all the required files:

imap_htdocs.zip (8kB)

You can use the files as a jumping board or to set up the website on an AWS server do the following:

  • launch a new AWS Instance using the latest Amazon Linux AMI
  • SSH to the new box
  • get root power:
    sudo su
    
  • install MySQL: follow the instructions in How to Install MySQL on an AWS Instance or:
    yum -y install mysql mysql-server
    chgrp -R mysql /var/lib/mysql
    chmod -R 770 /var/lib/mysql
    chkconfig --levels 235 mysqld on
    service mysqld start
    mysqladmin -u root password 0123456789
    
  • install Apache/PHP: follow the instructions in How to Install Apache/PHP on an AWS Instance or:
    yum -y install php php-mysql httpd
    chkconfig --levels 235 httpd on
    service httpd start
    
  • open up port 80 inbound in your AWS Security Group (from MyIP is more secure)
  • we will also be needing the PHP Imap extensions, install with:
    yum -y install php-imap
    service httpd restart
    
  • download and install the above zip to the box by running
    # move to webroot
    cd /var/www/html
    # download the zip
    wget http://www.quickstepapps.com/wp-content/uploads/2014/10/imap_htdocs.zip
    # unzip it
    unzip imap_htdocs.zip
    # delete zip
    rm -f imap_htdocs.zip
    # set webroot permissions
    find /var/www/html -type d -exec chown root:apache {} +
    find /var/www/html -type d -exec chmod 550 {} +
    find /var/www/html -type f -exec chown root:apache {} +
    find /var/www/html -type f -exec chmod 440 {} +
    # write access to attachments directory
    chmod 770 /var/www/html/attachments
    
  • install the database with:
    mysql --host=localhost --user=root --password=0123456789 --execute="source db.sql"
    # delete the script (if you want)
    rm -f db.sql
    
  • enter your Gmail credentials in the file sched/fetchemails.php with your favourite editor. This is in the two variables gmail_username and gmail_password right at the top.
  • now if you point your browser to your Public IP, you should see the password prompt: the password is 'admin'.

Test the system by sending an email to your Gmail address (or company address that is forwarding to Gmail) with an image attached. Then Click 'Check Now' on the Admin Website. When it's done, Click 'Emails' on the original page and you should see your email and image!

If you wanted to schedule the fetchemails.php page to be called every hour, you could use:

line="0 * * * * wget -O - http://localhost/sched/fetchemails.php"
(crontab -u root -l; echo "$line" ) | crontab -u root -

Don’t you just love email?

Leave a Reply

Your email address will not be published. Required fields are marked *