WarpConduit Computing

Preserve your MySQL Databases Between Destroy and Up with Vagrant + PuPHPet on Windows

by Leave a Comment

Have you ever run vagrant destroy on your box before you realized you wanted to keep the MySQL databases? Well I have, last night actually. Thankfully I use the wonderful and free Veeam Endpoint Backup to backup daily and was able to restore the old VMDK, connect it to a VirtualBox VM, use Ubuntu recovery mode to gain access, and then backup the databases to a shared folder. So that was an exciting learning experience.

Problem

And so it got me thinking – how can I make the databases persist between vagrant destroy and vagrant up so that I don’t lose them just because my memory was lacking. Separate VM? No, that’s more resources and adds complexity. Map MySQL data directory to host? No, been there done that, DB access is noticeably slower. Forget Vagrant and use Docker? No, I’ve tried it and though many love Docker it’s not for me; perhaps when Docker for Windows is as good as Docker for Mac/Linux.

Solution

Use the vagrant-triggers plugin and a couple bash scripts to obtain developer happiness.

Prerequisites

Getting Started

Important: Use an elevated Command Prompt (Start button, type “cmd”, hold Ctrl+Shift and press Enter) for all commands.

Create a new folder where you would like to store the content and config for your new Vagrant box. In my case I have a Vagrant folder on a secondary hard drive that I use for Vagrant boxes. The directory name you choose isn’t important as long as you remember what box is in it. The Vagrant box’s actual name will be taken from PuPHPet’s config.yaml regardless.

C:\WINDOWS\system32>d:
D:\>cd Vagrant
D:\Vagrant>mkdir Developer
D:\Vagrant>cd Developer
D:\Vagrant\Developer>

Extract the customized Vagrant + PuPHPet config (see Prerequisites above) there and you should have the following files present:

D:\Vagrant\Developer>dir
 Volume in drive D is Storage
 Volume Serial Number is 4051-68B4
 
 Directory of D:\Vagrant\Developer
 
02/10/2017  04:54 PM    <DIR>          .
02/10/2017  04:54 PM    <DIR>          ..
02/10/2017  04:36 PM               135 .gitattributes
02/10/2017  04:36 PM                27 .gitignore
02/10/2017  04:37 PM    <DIR>          puphpet
02/10/2017  04:36 PM               861 Vagrantfile
02/10/2017  04:42 PM    <DIR>          wincachegrind
02/10/2017  04:43 PM    <DIR>          www
               3 File(s)          1,023 bytes
               5 Dir(s)  287,536,390,144 bytes free

The www directory will contain web content, xdebug output files, and MySQL database backups. The wincachegrind directory contains WinCacheGrind, a viewer for xdebug output files.

Issue a vagrant up and take a break while the box is provisioned.

D:\Vagrant\Developer>vagrant up
Bringing machine 'developer' up with 'virtualbox' provider...
==> developer: Importing base box 'puphpet/ubuntu1604-x64'...
==> developer: Matching MAC address for NAT networking...
==> developer: Checking if box 'puphpet/ubuntu1604-x64' is up to date...
==> developer: Setting the name of the VM: Developer_developer_1486767926284_24830
==> developer: Clearing any previously set network interfaces...
==> developer: Preparing network interfaces based on configuration...
    developer: Adapter 1: nat
    developer: Adapter 2: hostonly
==> developer: Forwarding ports...
    developer: 22 (guest) => 6241 (host) (adapter 1)
    developer: 22 (guest) => 2222 (host) (adapter 1)
==> developer: Running 'pre-boot' VM customizations...
==> developer: Booting VM...
==> developer: Waiting for machine to boot. This may take a few minutes...
 
...
 
==> developer: Did something go wrong? Don't worry! I can (maybe) help!
==> developer: Please go to our Github issues page at:
==> developer:
==> developer: https://github.com/puphpet/puphpet/issues
==> developer:
==> developer: and search for your problem. If you do not find your problem
==> developer: answered, open a new ticket!
==> developer:
==> developer: ------------------------------------------------------------
==> developer: PLEASE REMEMBER TO INCLUDE THE CONTENTS OF YOUR
==> developer: "puphpet/config.yaml" FILE.
==> developer: ------------------------------------------------------------
==> developer:
==> developer: Make sure to xxx out any potential API keys or passwords
==> developer: that you do not want others to see!
==> developer:
==> developer: Happy programming!
==> developer:                                              - Juan Treminio
==> developer:
==> developer: ++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++
==> developer: |  ____                _      _    _                    _  |
==> developer: | |  _ \ ___  __ _  __| |    / \  | |__   _____   _____| | |
==> developer: | | |_) / _ \/ _` |/ _` |   / _ \ | '_ \ / _ \ \ / / _ \ | |
==> developer: | |  _ <  __/ (_| | (_| |  / ___ \| |_) | (_) \ V /  __/_| |
==> developer: | |_| \_\___|\__,_|\__,_| /_/   \_\_.__/ \___/ \_/ \___(_) |
==> developer: |                                                          |
==> developer: ++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++

Alright, if everything went smoothly you should have a working vagrant box at http://developer.test that displays the PHP info page.

At this point we have no MySQL data but we do have an empty database, db1, so let’s install WordPress to generate some data.

After extracting the WordPress install zip file to D:\Vagrant\Developer\www\developer.test\public_html we can start the install at http://developer.test/wordpress.

Okay, done!

Destroy! Destroy!

Now let’s perform a destroy and see what happens.

Important: The vagrant box must be running when you issue the destroy command because the database backup script needs to run inside box. Otherwise you’ll get a “not yet ready for SSH” error and return to the command prompt.

D:\Vagrant\Developer>vagrant destroy
==> developer: Running triggers before destroy...
==> developer: Dumping the databases before destroying the VM...
==> developer: Executing remote command "cd /var/www/mysql && bash backupdbs.sh"...
==> developer: mysql: [Warning] Using a password on the command line interface can be insecure.
==> developer: Dumping database: db1
==> developer: mysqldump: [Warning] Using a password on the command line interface can be insecure.
==> developer: Remote command execution finished.
    developer: Are you sure you want to destroy the 'developer' VM? [y/N] y
==> developer: Forcing shutdown of VM...
==> developer: Destroying VM and associated drives...
==> developer: Pruning invalid NFS exports.
==> developer: [vagrant-hostsupdater] Removing hosts

Nice, and what do we find in D:\Vagrant\Developer\www\mysql now?

D:\Vagrant\Developer\www\mysql>dir
 Volume in drive D is Storage
 Volume Serial Number is 4051-68B4
 
 Directory of D:\Vagrant\Developer\www\mysql
 
02/10/2017  07:19 PM    <DIR>          .
02/10/2017  07:19 PM    <DIR>          ..
02/10/2017  07:19 PM            38,390 backupdbs.db1.sql
02/10/2017  03:59 AM               443 backupdbs.sh
02/10/2017  03:59 AM               180 restoredbs.sh
               3 File(s)         39,013 bytes
               2 Dir(s)  287,498,833,920 bytes free

Yes, our database has been preserved!

Up Vagrant, Up!

Let’s provision this VM again and see what happens near the end of the process:

D:\Vagrant\Developer>vagrant up
...
 
==> developer: Running files in files/startup-once
==> developer: Created file /.puphpet-stuff/startup-once-ran
==> developer: Importing backupdbs.db1.sql ...
==> developer: mysql: [Warning] Using a password on the command line interface can be insecure.
==> developer: Finished running files in files/startup-once
 
...

And if we visit http://developer.test/wordpress after destroying and provisioning the box – no difference – success!

How does it work?

Here are the changes that were required to backup the databases before a “destroy” and restore after the first “up”.

First, there is a bit of code added to Vagrantfile in order to trigger a database backup before “destroy”.

In D:\Vagrant\Developer\Vagrantfile you’ll find this section:

Vagrant.configure('2') do |config|
  eval File.read("#{dir}/puphpet/vagrant/Vagrantfile-#{data['target']}")
end

And the updated code adds a trigger before the end:

Vagrant.configure('2') do |config|
  eval File.read("#{dir}/puphpet/vagrant/Vagrantfile-#{data['target']}")
 
  # BACKUP MYSQL DATABASES
  config.trigger.before :destroy do
    info "Dumping the databases before destroying the VM..."
    run_remote  "cd /var/www/mysql && bash backupdbs.sh"
  end
 
end

And D:\Vagrant\Developer\www\mysql\backupdbs.sh contains:

#!/bin/bash
 
USER="root"
PASSWORD="1234"
 
databases=`mysql -u $USER -p$PASSWORD -e "SHOW DATABASES;" | tr -d "| " | grep -v Database`
 
for db in $databases; do
    if [[ "$db" != "information_schema" ]] && [[ "$db" != "performance_schema" ]] && [[ "$db" != "sys" ]] && [[ "$db" != "mysql" ]] && [[ "$db" != _* ]] ; then
        echo "Dumping database: $db"
        mysqldump -u $USER -p$PASSWORD --databases $db > backupdbs.$db.sql
    fi
done

It grabs a list of user databases and dumps each one to a SQL file. That takes care of the backup. The restore is triggered by PuPHPet on the initial “up” of the vagrant box.

D:\Vagrant\Developer\puphpet\files\startup-once\restoredbs.sh contains:

cd /var/www/mysql
bash restoredbs.sh

And D:\Vagrant\Developer\www\mysql\restoredbs.sh contains:

#!/bin/bash
 
USER="root"
PASSWORD="1234"
 
databases=`ls -1 backupdbs.*.sql`
 
for db in $databases; do
        echo "Importing $db ..."
        mysql -u $USER -p$PASSWORD < $db
done

It grabs a list of SQL files created by the backup and imports each one into the MySQL server.

Important: If you are copying code from this page and creating your own script files make sure the line endings are set to UNIX (LF only). Otherwise Vagrant will fail to execute them.

Conclusion

Not only does this preserve your MySQL databases but with this method also opens up the possibility of distributing a development environment with applications and databases already installed and working from the first startup of your Vagrant box.

Hope this article has been useful to you, drop a comment if you feel so inclined.

Configuring Exim for Authenticated SMTP Relay to Gmail on CentOS/RHEL

by 2 Comments

exim-gmail-smtp-relayI’ll start this by saying that Exim is slightly more difficult to configure for Gmail SMTP relay than Postfix, and if you’re using Webmin there is a standard module for Postfix that makes it even easier. Finding the correct way to configure Exim on CentOS was difficult in itself and the reason for this article.

Now, if you prefer to use Exim for sending mail through Gmail this is how you accomplish it.

These instructions have been tested on CentOS 6.7 with Exim 4.84.

Preparation

Log in to the Google/Gmail account you will be using for relaying and enable access from less secure apps in the security options. If you don’t make this change you will not be able to authenticate with Gmail to send mail. From my experience, Postfix and Exim both require this.

If you have Postfix or another MTA already installed you’ll want to remove them or disable them.

Next, make sure you have Exim and the SASL authentication methods installed by running:

yum install -y exim cronie cyrus-sasl cyrus-sasl-plain

Note: I’ve included cronie since it gets removed if you uninstall Postfix and other MTAs.

If that completed without errors you’re ready to continue.

Configuration

Using your favorite editor, open /etc/exim/exim.conf and find the following sections:

begin routers

Below it, add these lines:

send_via_gmail:
 driver = manualroute
 domains = ! +local_domains
 transport = gmail_smtp
 route_list = * smtp.gmail.com

Continue to the next section:

begin transports

Below it, add these lines:

gmail_smtp:
   driver = smtp
   port = 587
   hosts_require_auth = *
   hosts_require_tls = *

Continue to the next section:

begin authenticators

Below it, add these lines:

gmail_login:
   driver = plaintext
   public_name = LOGIN
   client_send = : user@gmail.com : myPassw0rd

Replace the credentials with your actual credentials on the client_send line.

That’s it for the configuration. If you started with a default exim.conf then your updated configuration file will look like this gist.

Restart Exim to make these changes live:

service exim restart

Testing & Troubleshooting

Test by sending an email with echo "Just testing." | mail -s "Test Subject" user@example.org or a simple PHP script. Use a destination address separate from the one you’re using to relay email.

Shortly thereafter you should receive the email at the destination mailbox (check the spam folder too). If you don’t receive it then check the log at /var/log/maillog or /var/log/exim/main.log for errors.

If you get an error from Gmail, they usually reference a URL in the error that you can visit for more info. You can also do a Google search for the error to see what others have done to resolve it.

Of course, the hope is that you didn’t have any errors and you’re all set to send emails from your Linux box. Enjoy!

As a side note, if you are using Webmin there is a third-party Exim module that allows you to view the Exim logs, queue, and statistics.

Conclusion

Perhaps you found this article useful or have an issue that you couldn’t resolve with a Google search? Leave a comment and see what becomes of it.

Adding Custom Fonts to the Beaver Builder Child Theme Customizer

by

Recently I was searching for an easy way for developers to add custom fonts to the Beaver Builder child theme customizer. Failing to find that, I have come up with a solution that does not modify the core Beaver Builder theme files and only requires a minimal amount code.

Step 1: Load Your Font

You have a few options and the method will depend on your font source. Perhaps it is a kit from Typekit or Edge Web Fonts loaded via JavaScript. In that case you can add the code through Customize … Code > Head Code.

bb-customize-head-code

Alternatively, you could enqueue the embed code through the child theme’s functions.php file, it’s entirely up to you.

Some other methods include uploading your custom CSS font files to your child theme’s directory or using a remote @import URL. Either way you’ll need to edit your child theme’s style.css file and load the font from there.

Important Note: Currently, the Beaver Builder theme code doesn’t quote the font name in the generated CSS so for sure font names with commas will fail and you’re mileage may vary with fonts that include spaces or special characters. I have found that at least fonts with single spaces should work in all modern browsers without quotes.

Once you have your font loaded and can verify that in your site’s source code you can move on.

Step 2: Add Your Font to the Theme Customizer Font List

Edit your child theme’s functions.php file and add the following, making sure to replace acumin-pro-extra-condensed with your custom font name and adjusting the fallback string and weights array as needed. You can add multiple custom fonts to the $custom_fonts array, just follow the same format as the first.

add_action( 'init', 'customize_font_list' );
function customize_font_list(){
 
    $custom_fonts = array(
        'acumin-pro-extra-condensed' => array(
            'fallback' => 'Arial, sans-serif',
            'weights' => array(
                '400',
                '700'
            )
        )
    );
 
    foreach($custom_fonts as $name => $settings){
        // Add to Theme Customizer
        if(class_exists('FLFontFamilies') && isset(FLFontFamilies::$system)){
            FLFontFamilies::$system[$name] = $settings;
        }
 
        // Add to Page Builder
        if(class_exists('FLBuilderFontFamilies') && isset(FLBuilderFontFamilies::$system)){
            FLBuilderFontFamilies::$system[$name] = $settings;
        }
    }
}

Step 3: Choose Your Font in the Theme Customizer or Page Builder

In the Theme Customizer navigate to a section with a Font Family dropdown such as General > Headings.

As long as your custom CSS font is loading and you have no errors in your PHP code you should now see your custom font name listed under the System group of the dropdown list.

bb-customize-headings

Theme Customizer

bb-page-builder-heading

Page Builder

Hope this technique is helpful to you, leave your comments and questions below.

Custom WordPress Dashboard Widgets with Configurable Content

by

After working with WordPress dashboard widgets in a recent project I wanted to present a complete example with arguments and usage of the control callback for saving widget options. Using the code in the Gist below you will be able to configure custom content for this widget with the native WYSIWYG editor. When you hover over the widget’s title bar the “Configure” link will be revealed – you must be an Administrator to configure the widget.

custom-dashboard-widget-01

Widget Display

custom-dashboard-widget-02

Widget Configuration

There are couple of special things to note. The first is the workaround to enable the autoembed functions for content output (lines 12-16). The dashboard has no post ID and without a post ID that evaluates to true autoembed will fail to do its job. The second thing to note is code to set the widget priority (lines 81-85). However, once a user manually positions a dashboard widget this code is no longer effective.

You may find this code useful for developing a “getting started” widget to help users use custom WordPress admin features you’ve developed.