Building the back end for a webmail application

Updated February 9, 2019

Mailgun offers an API to create email apps with 10,000 free emails per month. Instead of using existing webmail apps, let's create a custom webmail client for Mailgun.

We'll use Visual Studio and C# for development, using Web API for its simplicity, and .NET Core for it's improved performance and cross-platform capabilities. First, we'll launch an application server running .NET Core in AWS. Then, we'll register a domain, configure Mailgun, and install dependencies. We'll end part 1 by deploying and testing a webhook to receive emails.

1. Launch a .NET Core server with AWS EC2

Create an AWS account if you don't already have one. Once you're logged in to AWS, go to your EC2 dashboard.

In your EC2 console, click on Launch Instance.

On Step 1: Choose an Amazon Machine Image (AMI), click on the Community AMIs tab on the left, and search for net core. From the results, select Amazon Linux 2 LTS AMI with .NET Core 2.1.

Steps 2 - 5 are mostly related to hardware, and for this project, the default settings are sufficient.

On Step 6: Configure Security Group, let's create a new security group called MailServerACL with the following rules:

  • Open access to HTTP on port 80
  • Open access to HTTPS on port 443
  • Open access to Kestrel on port 5000
  • Limited access to SSH on port 22
  • Limited access to MySQL on port 3306

On Step 7: Review Instance Launch, click on Launch. At the prompt, download the private keyfile assigned to your instance. We'll use this key to connect to our server.

Once your instance is running, write down the IPv4 address of your instance. We will be using this value later.

Maintenance Tip

If you decide to restart your instance at a later time, the IPv4 address assigned to the instance may change. To prevent this, you may attach an elastic IP to your instance. Elastic IPs are static IPv4 addresses that you can allocate from Amazon, helping you avoid IP configuration changes if you decide to restart your instance at a later time.

2. Domain registration with AWS Route53

Back in your AWS console, go to your Route 53 dashboard and register a domain name. I will be using cratemail.org for this tutorial.

Domain registration can take up to an hour. When your domain is ready, it will appear under Hosted Zones.

In your domain's hosted zone, you should be able to create, modify, and delete DNS records. Let's connect your domain to the server. Click on Create Record Set.

On the right-hand panel, under type, select A - IPv4 address. For the value, add your instance's IPv4 address. Click on Create at the bottom to connect your domain.

3. Mailgun Configuration

Create a Mailgun account if you don't have one. Once you're logged in to Mailgun, go to your dashboard. Under the Domains tab, click on Add New Domain and type in your domain name. To use Mailgun, you will have to copy the presented DNS records to your domain's hosted zone. There are 3 record sets to create.

The first record is a general SPF record. Click on Create Record Set, and enter the following text record set.

The second record is a DKIM record with a unique key value for your account.

The third record is a multiple value MX record.

Go back to Mailgun's domain configuration page and click on Check DNS Records Now. If the record sets were configured properly, your domain will go into Active state, meaning it will be able to send and receive emails using your domain name.

Finally, we will want to route incoming emails to our webmail app through an API endpoint. Under the Routes tab, click on Create Route, and create a route with the following settings:

  • Expression Type: Catch All
  • Under Actions, check "Store and notify". Add https://cratemail.org/api/Mail to the field, replacing cratemail.org with your domain name

4. Web server setup

4.1 Connect to .NET Core web server

Download PuTTY, an SSH client, to securely access the web server using your private keyfile.

Launch a PuTTY session with the following configuration:

  • 1: Under Session, type in your domain name for the Host Name (or IP address) field.
  • 2: Under Connection -> SSH -> Auth, on the last field, browse for your private keyfile.
  • Under Connection -> Data, under login details, type in ec2-user for auto-login username
  • 3,4: (Optional) To save your configuration, type in a name under Saved Sessions and click on Save.

4.2 Install Caddy

Caddy is a modern HTTP/2 web server with automatic HTTPS, built by Matt Holt. We'll need this to create a secure environment for our webmail operations. Run the following commands to set up Caddy:

  • sudo -i to get admin privileges.
  • cd /home/ec2-user to go to the home directory.
  • wget https://github.com/mholt/caddy/releases/download/v0.11.0/caddy_v0.11.0_linux_amd64.tar.gz to download Caddy.
  • tar -xf caddy_v0.11.0_linux_amd64.tar.gz caddy to unzip our Caddy distribution.
  • nano Caddyfile to create a configuration file for our Caddy web server with the following content:

  • cratemail.org {
         proxy / localhost:5000 { 
              transparent 
         } 
    }

    Here, we are telling Caddy to redirect web visitors to Kestrel, the integrated web server for .NET Core. Don't forget to replace cratemail.org with your domain name. Press Ctrl + O, then enter, to save the Caddyfile. Finally, run the following command to launch Caddy:

    • ./caddy

Close the shell, and launch another PuTTY window to install MariaDB.

4.3 Install MariaDB

MariaDB is a modern port of MySQL. We'll use it to store email data for our webmail app. Run the following commands to set up MariaDB:

  • sudo -i to get admin privileges.
  • yum install -y mariadb-server to install MariaDB from Amazon's repository.
  • systemctl start mariadb to launch MariaDB.
  • mysql_secure_installation you will be prompted with a set of questions. Remember your SQL password, we will use this later.
  • nano /etc/my.cnf to modify MariaDB's configuration file.
  • Type in a hashtag before bind-address = <some ip-address> to comment this line out. Skip this step if you don't find the line, otherwise, comment and press Ctrl + O, then enter, to save the MariaDB configuration file.
  • mysql -p to run SQL commands.
  • GRANT ALL PRIVILEGES ON *.* TO 'root'@'localhost' IDENTIFIED BY 'mypass' WITH GRANT OPTION;, replace 'mypass' with your SQL password.
  • GRANT ALL PRIVILEGES ON *.* TO 'root'@'1.2.3.4' IDENTIFIED BY 'mypass' WITH GRANT OPTION;, replace '1.2.3.4' with your IP address, and 'mypass' with your SQL password.
  • CREATE DATABASE cratemail to create a database for our email app.
  • systemctl restart mariadb to restart MariaDB with our recent changes.

With the web server and database ready, we can begin to develop our webmail application in Visual Studio.

5. Create ASP.NET Core Web app

Download and install the latest version of Visual Studio. It should come with the latest version of .NET Core.

In Visual Studio, create an ASP.NET Core Web Application project. On the launch window, select ASP.NET Core 2.2 from the top dropdown. Select React.js and Redux from the project templates, and uncheck Configure for HTTPS. We will let Caddy act as our HTTPS server instead. Click OK to create the project.

5.1 Download NuGet packages

Go to Tools -> NuGet Package Manager -> Manage NuGet Packages for Solution. Click on the Browse tab to search for packages.
We will need 2 packages for our project. The first package is an Entity Framework Core driver for MySQL, compatible with MariaDB, which we'll use to modify our database from Visual Studio. Search for Pomelo.EntityFrameworkCore.MySql, then select it from the list, then select your project on the right hand panel, and click Install.
The second package is RestSharp, a simple to use HTTP client for C#. Search for RestSharp and install this package into your project as well.

5.2 Save connection string and Mailgun key

For coding convenience, let's store our connection string and Mailgun key as text resources in our project. In the Solution Explorer window, right-click on your project and choose Properties. Under Resources, click on the link to generate a resource file.

Add a string resource named connString. The value should be your MariaDB connection string using the following format:

server=cratemail.org;database=cratemail;user=root;password=mypass

Add a string resource named mailgunKey. The value should be your Mailgun API key, which you can retrieve from the domain details page in your Mailgun dashboard.

5.3 Using Entity Framework Core to create Mail table

Let's create an entity to store mail data. In Solution Explorer, right-click on your project, and add a new folder called Model. Then right-click on the Model folder, go to Add -> and select Class... from the bottom of the list. Name this class Mail.cs and add the following content:

namespace Cratemail.Model
{
    public class Mail
    {
        public int ID { getset; }
        public string Sender { getset; }
        public string Recipient { getset; }
        public string MailSubject { getset; }
        public string BodyHtml { getset; }
        public string BodyPlain { getset; }
        public string MailDate { getset; }
    }
}

Next, create another class under the Model folder, and call it Context.cs, with the following content:

using Microsoft.EntityFrameworkCore;
using Cratemail.Properties;

namespace Cratemail.Model
{
    public class ContextDbContext
    {
        public DbSet<Mail> Mail { getset; }
        protected override void OnConfiguring(DbContextOptionsBuilder optionsBuilder)
        {
            optionsBuilder.UseMySql(Resources.connString);
        }
    }
}

With your context class ready, go to the Package Manager Console by clicking on View -> Other Windows -> Package Manager Console.

At the PM> prompt, type in Add-Migration InitialDB and press enter. This will generate a migration file with our new entity. Back at the prompt, type in Update-Database and press enter to sync with MariaDB.

5.4 Verifying database changes with HeidiSQL

HeidiSQL is a graphical client for MariaDB. Download and launch HeidiSQL, then create a session with the following configuration:

Under "Settings" tab:

  • For network type, select MySQL (SSH tunnel).
  • For Hostname/IP, type in localhost.
  • For User, type in root.
  • For Password, type in your MariaDB password.

Under "SSH tunnel" tab:

  • For plink.exe location, download plink and type in the location of the plink.exe file in your filesystem.
  • For SSH host + port, type in your domain name for the host portion, and 22 for the port portion.
  • For Username, type in ec2-user.
  • For Private key file, enter your private key.

Save your session, then click Open to connect to MariaDB. On the left panel, double-click on the cratemail database. You should be able to see a Mail table. Double-click on the Mail table to verify that the properties in our Mail.cs class propagated as columns in the table.

6. Build API endpoint

To store incoming email, we'll have to build an API endpoint for the Mailgun route we configured, /api/Mail. Mailgun will route all incoming emails to your API through an HTTP POST request with message parameters. Let's encapsulate these parameters. Right-click on the Model folder, and add a class called Mailgun.cs with the following content:

using Microsoft.AspNetCore.Mvc;

namespace Cratemail.Model
{
    public class Mailgun
    {
        public string Recipient { getset; }
        public string Sender { getset; }
        public string From { getset; }
        public string Subject { getset; }
        [BindProperty(Name = "body-plain")]
        public string BodyPlain { getset; }
        [BindProperty(Name = "body-html")]
        public string BodyHtml { getset; }
    }
}

On the Solution Explorer, right-click on the Controllers folder, go to Add -> and click on Controller... on the top of the menu. Choose API Controller with read/write actions, and click Add.

At the prompt, name your controller MailController.cs, and click Add
To store incoming messages to MariaDB, add the following content to your controller:

using System;
using Microsoft.AspNetCore.Mvc;
using MySql.Data.MySqlClient;
using Cratemail.Model;
using Cratemail.Properties;

namespace Cratemail.Controllers
{
    [ApiController]
    [Produces("application/json")]
    [Route("api/Mail")]
    public class MailController : ControllerBase
    {
        [HttpPost]
        public IActionResult Post([FromForm]Mailgun msg)
        {
            string insertSql = "INSERT INTO Mail(Sender, Recipient, MailSubject, BodyHtml, BodyPlain, MailDate) VALUES (@Sender,@Recipient,@MailSubject,@BodyHtml,@BodyPlain,'{0:g}')";
            DateTime timestamp = TimeZoneInfo.ConvertTimeBySystemTimeZoneId(DateTime.Now, "US/Pacific");
            string insertQuery = String.Format(insertSql, timestamp);
            MySqlConnection sqlConnection = new MySqlConnection(Resources.connString);
            sqlConnection.Open();
            MySqlCommand sqlCommand = new MySqlCommand(insertQuery, sqlConnection);
            sqlCommand.Parameters.AddWithValue("@Sender", msg.From);
            sqlCommand.Parameters.AddWithValue("@Recipient", msg.Recipient);
            sqlCommand.Parameters.AddWithValue("@MailSubject", msg.Subject);
            sqlCommand.Parameters.AddWithValue("@BodyHtml", msg.BodyHtml);
            sqlCommand.Parameters.AddWithValue("@BodyPlain", msg.BodyPlain);
            sqlCommand.ExecuteNonQuery();
            sqlConnection.Close();
            return CreatedAtAction(nameof(Post), msg);
        }
    }
}

Press Ctrl + Shift + B to build your solution.

7. Upload app

Let's test our newly created API controller by uploading it to our EC2 instance. To do this, download and install WinSCP. Launch WinSCP with the following configuration:

  • For File protocol, choose SFTP.
  • For Host name, type in your domain name.
  • For port number, type in 22.
  • For User name, type in ec2-user.
  • Click on Advanced, and under SSH -> Authentication, enter your private key.

Once logged in, locate your Cratemail project folder on the left panel, and transfer it to the /home/ec2-user/ directory on the right panel.

When finished, open PuTTY and connect to your instance, then run the following commands:

  • sudo -i to get admin privileges.
  • cd /home/ec2-user/ to go into the home directory.
  • wget https://nodejs.org/dist/v8.11.4/node-v8.11.4-linux-x64.tar.xz to download NodeJS.
  • tar -xf node-v8.11.4-linux-x64.tar.xz to unpack our NodeJS distribution.
  • mv node-v8.11.4-linux-x64 nodejs to rename our NodeJS folder.
  • export PATH=$PATH:/home/ec2-user/nodejs/bin to add NodeJS to our environment path.
  • cd /home/ec2-user/Cratemail/ClientApp to navigate to the frontend portion of our project directory.
  • npm install to setup our frontend dependencies.
  • cd /home/ec2-user/Cratemail/ to go back into our project directory.
  • dotnet restore to prepare our project for launch.
  • dotnet run to launch our project.

If everything went well, you should see the following message on your shell:

Test your API endpoint by sending an email to any recipient in your domain. After sending the email, open HeidiSQL, go to the Mail table, and click on the Data tab. You should see a new row with the contents of your test email.

Go to my next post to continue building the frontend.