Category: DigitalOcean

Send Apache Logs to Papertrail With Rsyslog

Over the last few days, I’ve been looking at Apache web server logs, a lot, mostly quick checks for Shellshock probes and exploit attempts. All on client servers, thankfully. All of the servers I operate through DigitalOcean are patched up. It just so happens that all the sites I host have their DNS hosted by Cloudflare, which has been blocking all Shellshock attempts.

A majority of my sites send their Apache logs to Papertrail. Having all my apache logs easily accessible and searchable is extremely nice. It’d make sniffing out Shellshock attempts quite simple. You can check for Shellshock attempts relatively easily from the command line, as well, something like the command below would work:

grep '() {' /var/log/apache2/access.log

1. Setup Rsyslog to Send to Papertrail

Anyway, sending Apache logs to Papertrail is pretty easy. I’m going to assume you’ve already setup rsyslog to send logs to Papertrail. If not, this post should help.

2. Add CustomLog Directive To Your VirtualHost

You just need to modify your virtualhost configuration and add a CustomLog directive. Here’s what I do to send longren.io logs to Papertrail:

CustomLog "| /usr/bin/logger -t httpd -p local1.info" "%{%b %d %X}t longren.io apache %h %l %u %t "%r"%>s %b "%{Referer}i" "%{User-agent}i""

The -t httpd piece sets the service name for Papertrail. The -p local1.info flag sets the priority. You’ll want to change the longren.io piece in the above code to whatever site you’re capturing logs for. You can also change or remove apache that immediately follows longren.io.

3. Reload Apache

After you’ve added the CustomLog directive to your virtualhost, you’ll want to reload Apache:

sudo service apache reload

That’s all there is to it. You should start seeing your Apache logs in Papertrail shortly after reloading Apache.

0

TinyCert: Generate SSL Certificates And Become Your Own Certificate Authority

A few days ago I moved longren.io to https. I didn’t pay for a certificate though like I would when setting up an e-commerce site or something else important.

I even get the little green lock symbol in the address bar, but I think this is mostly due to my use of Cloudflare.

TinyCert is a service I discovered that lets you be your own PKI/certificate authority. It’s entirely free and provides you with a very nice interface for managing your certificates. The image below shows the interface for managing your certificates. The list on the right is a list of certificates, as you can see I’ve got one made up for longrendev.io, but haven’t put it in place quite yet.
tinycertinterface

The support from TinyCert is very good as well, I had a few questions regarding how their certificates would work with Cloudflare and they quickly cleared my questions up. SSL Labs from Qualys gives the SSL certificate an “A” rating. Should you use certificates from TinyCert in production? Probably not. I am, however, due to my use of Cloudflare.
ssl

This post isn’t meant to show you how to install certificates or use TinyCert, it’s simply to make you aware of the tool and what can be done with it. TinyCert has a pretty extensive FAQ, so should you have questions, which I’m sure you do, head on over and start reading. If you do need help installing the certificates from TinyCert, their help center does a nice job of providing instructions for Apache and Nginx based setups.

Have fun with TinyCert, it’s a pretty awesome service that I’ll continue to use and will absolutely be donating to. But please remember, TinyCert certificates should not be used for regular public websites and the service is not a substitute for a proper certification authority, but for self-signed certificates.

0

Automate Taking Snapshots of Your DigitalOcean Droplets with DOSnapshot

Multi-threading. Auto-cleanup. Cron optimized.

There are a lot of neat tools people have built for DigitalOcean.

The app I’m really in love with is DOSnapshot, and is hosted on GitHub. DOSnapshot does exactly what its name would suggest, it takes snapshots of your droplets.

As of this post, I’m the only one that’s left a comment on the DOSnapshot Community Projects page, which took me a bit by surprise, given the quality of the tool.

Taking a snapshot of a DigitalOcean Droplet is essentially like making an exact copy of the Droplet (server) that you can then use again at a later time. Very useful for scaling and updating a Droplet to a newer version of your Linux distribution without losing all of the Droplet’s configuration.

Etel Sverdlov does a very good job of explaining the difference between snapshots and backups in this DigitalOcean community tutorial. I suggest you read it if you’re unsure what the differences between a backup and snapshot are.

1. Install DOSnapshot

DOSnapshot can be installed as a ruby gem, which is what I chose to do because it’s just so easy. Don’t install this on your DigitalOcean Droplet! It’s meant to run from your local machine. Installing DOSnapshot as a Rubygem is as simple as: 

sudo gem install do_snapshot

Pre-built binaries are also provided for Linux users, and OSX users have the option of installing via Homebrew Tap.

2. Set Your DigitalOcean Client ID and API Key

Once you’ve got it installed, you’ll need to set your DigitalOcean Client ID and API Key. You can set them as environment variables, or you can pass them as parameters when actually running DOSnapshot. This is straight from the README:

First you may need to set DigitalOcean API keys:

$ export DIGITAL_OCEAN_CLIENT_ID=”SOMEID”
$ export DIGITAL_OCEAN_API_KEY=”SOMEKEY”

If you want to set keys without environment, than set it via options when you run do_snapshot:

$ do_snapshot –digital-ocean-client-id YOURLONGAPICLIENTID –digital-ocean-api-key YOURLONGAPIKEY

3. Take A Snapshot

Please remember that running the do_snapshot command will cause your droplet to shutdown so the snapshot can be taken.

DOSnapshot has a pretty large number of options that you can specify. I’m going to keep this simple so you get the basics of it. Learning a few of the main options will be mostly what you need to know, after you’ve got them figured out, setting up a cronjob is cake.

You can take snapshots of all of your droplets at once, you can specify which droplets to take snapshots of, and you can specify droplets that you don’t want to take a snapshot of. I typically take a snapshot of a single droplet at a time, and I do it like this: 

do_snapshot --only 1111 -k 3 -c -v

The above will take a snapshot of only one droplet, a droplet with an ID of 1111, replace 1111 with the ID of your droplet. You can find your droplets ID in your browser URL bar while managing the droplet. So if you see https://cloud.digitalocean.com/droplets/1234567, your droplet’s ID is 1234567.

Here’s all of the options.

4. Scheduling With Cron

First, you must have cron installed. There’s plenty of tutorials on how to do that. That tutorial even explains how to configure a cron job using the crontab utility. There’s an example crontab entry in the DOSnapshot README. Mine is pretty simple: 

0 4 * * 2 do_snapshot --only 1111 -k 3 -c -v

If you have questions about setting any of this up, feel free to leave a comment!

0

Install Tarsnap On a DigitalOcean VPS or Any Ubuntu 14.04 or Newer System

Securely and remotely backup your server using Tarsnap

Tarsnap is a great service. It’s extremely affordable and secure. I mentioned it briefly in my previous post about MySQL backups.

Tarsnap is only $0.25 / GB-month for storage and $0.25 / GB for bandwidth, which is extremely affordable. You can add funds to your account whenever you like, which is also very nice. Your initial fund deposit must be at least $5. You can signup for a Tarsnap account here.

Installing it on a stock Ubuntu 14.04 LTS installation requires some additional steps to get everything working nicely.

1. Install dependencies

sudo apt-get install build-essential ext2fs-dev zlib1g-dev libssl-dev

2. Install Tarsnap

Download Tarsnap, I do it like this with wget:

wget --no-check-certificate https://www.tarsnap.com/download/tarsnap-autoconf-1.0.35.tgz

Now we need to extract, configure, and compile Tarsnap.

tar xfz tarsnap-autoconf-1.0.35.tgz
cd tarsnap-autoconf-1.0.35
./configure
sudo make install clean

3. Configure Tarsnap

First, copy the example config to the live config:

sudo mv /usr/local/etc/tarsnap.conf.sample /usr/local/etc/tarsnap.conf

If you receive an error with that command, like mkdir: cannot create directory , ignore it and continue on.

Next, we need to generate a tarsnap key for your machine. I like to keep my tarsnap key in my home directory, so I run something like this:

mkdir ~/.tarsnap
tarsnap-keygen --keyfile /home/youruser/.tarsnap/tarsnap.key --user your@email.com --machine your-machine-name

You’ll be prompted for your Tarsnap password when running tarsnap-keygen.

Now, edit the tarsnap.conf file:

sudo pico /usr/local/etc/tarsnap.conf

Point the keyfile directive to the key file we created a couple steps ago. The top of your tarsnap.conf file should look similar to this now:

### Recommended options

# Tarsnap cache directory
cachedir /tmp/tarsnap-cache

# Tarsnap key file
keyfile /home/youruser/.tarsnap/tarsnap.key

4. Use Tarsnap to Make a Backup

You’ll want get familiar with the Tarsnap manpages. To create your first archive, with a name of servername-20140805, do this:

tarsnap -c -f servername-20140805 /home/youruser

That will backup the /home/youruser folder to Tarsnap! Depending on the size of the backup and speed of your connection, it could take quite some time to finish the backup.

Below is the output of tarsnap –help, if you’re interested.
tarsnap-help

You can read more about Tarsnap at their homepage, tarsnap.com. They also have a page describing their infrastructure setup, which is kinda neat. Tarsnap also runs a bug bounty program.

I’ve only tested this on DigitalOcean VPS’s running Ubuntu 14.04 x64, but it should work on most Ubuntu variants.

Did I miss something or get something totally wrong? If so, please let me know, the comments are open!

0

Receive Alerts On SSH or SFTP Logins with Papertrail

Frustration-free log management, plus a lot more

I’ve been a huge fan of Papertrail ever since I discovered it, probably about a year ago or so. I use it mostly to monitor server logs. I currently have two servers setup to send syslog messages to Papertrail.

The Papertrail Events dashboard can be a bit overwhelming at first, but the provided search is powerful and allows you to finely control which log messages you see and which you don’t.

You can even setup saved searches to fire when a specific event occurs. For example, I have a saved search that searches for the following:
Accepted publickey for tyler

When that message shows up in Papertrail, it means that I logged in, or that someone else has logged in using my SSH key. This can be quite handy, especially if you’re a one man shop like me and are usually the only person that has SSH or SFTP access to a server.

Getting a DigitalOcean VPS added to Papertrail, especially if it’s running Debian or Ubuntu, is super easy. It just requires that you modify /etc/rsyslog.conf and add a line to the end of the file that will send a copy of the system logs to Papertrail.

Papertrail can monitor application logs, too, such as Apache httpd logs and MySQL server logs, although that takes a bit more configuration to get working properly.

If nothing else, it’s just nice having system logs aggregated in one central place, where everything is easy to search through, making it easy to find exactly what you’re looking for. If you’re an admin for one server or hundreds of servers, Papertrail could turn out to be one of your favorite tools. It’s definitely one of my favorites.

I suggest you give Papertrail a try, can’t hurt, they even have a plan that’s free forever. It’s definitely a great service for monitoring server logs.

0