Lately I’ve been working less on product development, and more on promoting my existing products, mostly my Android apps, but also my WordPress plugins. One of the apps seems like it is beginning to take off organically just by being on Google Play, but I feel that all of them could benefit from a little extra push.

I don’t hate marketing any more

As much as I don’t love marketing or marketers, after trying to do it myself, I learned a lot about what it entails, and I even gained a little appreciation towards the discipline (not a lot, mind you). Getting your product known to potential users is a necessity, and I now realize that what people mean when they say “I hate marketing”, is that they hate marketing that is done badly. With marketing, essentially, you’re just telling people “Hey, I made this, maybe you’re interested in it?” There’s nothing wrong with marketing when done right, as long as you don’t annoy people.

An example of right versus wrong kind of marketing: I see a lot of app developers fail because they use interstitial ads too much in their apps. These are the type of intrusive ads that have the highest payout, but also they annoy users the most (what with being intrusive and all). In my apps, I like to focus on serving banner ads, and only show interstitial ads for features that are beyond the main value proposition of my app. Interstitial ads are OK, if you first negotiate with your users, honestly and without deception, that they will sit through an annoying ad and then they’ll get something extra. Something that they didn’t expect to get when they downloaded the app. I’m not going to get any ad revenue if the user gets annoyed and uninstalls my app. On the other hand, if the app satisfies a need, a small banner at the bottom of the screen is not going to annoy users. A banner ad can generate significant revenue, once you hit a respectable number of users.

Some key insights

As for advertising my own apps, I’m only an amateur marketer, but already I’ve identified two key insights that are prevalent in the industry:

1. Video is king.

2. Only losers pay for sex and marketing.

The first one is obvious. The main message of your campaign should be available in all modes of media: text, images, and video. But video is king. It’s engaging. It’s rich. It sets its own pace. It stimulates two senses (sight and hearing). Reading is so last-century. Show-and-tell is more effective than a long article explaining things. Customers want solutions, not lectures. (OK, that’s actually a third insight!)

The second insight is less obvious. Paying for ads only works marginally well, once you have an excellent product and an excellent well-tuned ad. If you don’t know exactly what you’re doing, you can burn through your advertising budget pretty quickly with very poor results.

And if you don’t know what you’re doing, you don’t want to rely on a marketing agency in my opinion. That’s even less efficient than just paying for ads. It’s best to learn the skill yourself. I’m generally a DIY type of guy, I only rely on a professional for accounting.

DIY posting on social media, can be more effective: It’s free. It’s organic. You get analytics feedback that helps you learn the skill. Your viewers let their guard down and give a chance to your message. Banner blindness is a thing.

Shorts: not just a type of pants suitable for warm weather

Video shorts are vertical videos that have an aspect ratio of 9:16, and are typically 1080×1920 if you aim for HD. All video platforms now have them. They are suitable for viewing by teenagers, with a phone held vertically.

Every platform has its own set of technical requirements or recommendations. For compatibility with most platforms, I aim at 30 to 45 seconds. (Incidentally, this is the attention span of the average zoomer.)

Here’s a few more insights to consider, in no particular order:

• Start by writing the short video’s script before doing anything else. Use short sentences. Active voice. Simple vocabulary. You are speaking English potentially to people who struggle with English, or who are young, or tired, or distracted. But even your more sophisticated viewers will respond better to direct messages.
• Here’s a good structure: Start with a hook (why should the viewer even look at your video and not skip?) This will be a joke, something interesting, a problem they may be facing, an engaging meme or video clip, a face, something. This must be done in the first 2 or 3 seconds of the video. Then, take a couple of seconds to introduce the name of the product. Then, in the body of your message, showcase the product, illustrating exactly three (3) main points about it. Three is a good number. Then repeat the name of the product. Repetition is key. Finally, close by showing the product again, (remember, repetition is key), and add your call to action. What do you want the customer to do? Go to your website? Download the app? Contact you? Where? Make it easy and direct. Add all the recognizing features of your product there. Branding (colors, fonts, tagline, icon, any graphics) should all be prominent in the last scene. If you are offering something for free, such as in a freemium business model, make sure to make FREE the last word, because that’s what people will remember.
• There is going to be a voice talking. It’s better if it’s a female voice. Females sound attractive to your male audience, and appear less threatening to your female audience. Nobody wants to listen to a guy mansplaining your product.
• Voice is not enough. Many people have their phone’s volume on mute. Subtitles must be available. Every. Single. Word. Is. Its. Own. Separate. Subtitle. Subtitles must be near the center of the screen, but maybe shifted slightly to the bottom or top so as to leave space for you to showcase the product. Text must be in a heavy sans-serif fond, with bright color, and with a thin dark border. You want the text to be readable on all kinds of background. If the platform supports it, also add the subtitles as text, even though you’ve hard-coded them into the video. I heard people like subtitles, so I add subtitles to my subtitles.
• All platforms let you add some text to your video. Again, keep it short and to the point. Preferably the text should not deviate too much from the transcript of the video, which is your main message. Repeat your message there in the same template. Repetition is key.
• You can use memes to make your message more engaging. Memes are not copyrighted, but are easily recognizable. Adding an element of humor to your ad can go much further than spending a big budget on a marketing campaign.
• Graphics and animations are what make your video look professional. You can get graphics from stock image repositories. Be mindful of any copyrights.
• There is theory behind colors. Blue and white conveys seriousness, etc. Same for fonts, design, etc. Know when to use serif versus sans serif fonts. You don’t have to be a graphics designer, but you should take some time to study the basics. Be aware of visual hierarchy. Learn these things, and you’ll never have to talk to a “creative” person again in your life. Big plus!
• You should likely add some background music at low volume. The music should not overpower the speech or distract from your message. Viewers must subconsciously associate your product with positive, upbeat feelings. The music must be upbeat, and must strike a stark contrast to their otherwise miserable depressed life. The music must also be somewhat repetitive, so as not to distract from your messaging. Bonus points if it has a hypnotic element (ads work best by hypnotizing the logical faculties and appealing to emotion). The music must appeal to a wide array of musical tastes. Acoustic guitar riffs are the cornerstone of advertising. You can find royalty free music on pixabay. Be mindful of any usage rights. Sometimes the artist has submitted the music to a copyright ID platform, and you may have to request and submit usage rights. But things are simpler if you find music that is completely free to use.

Spam your ad to all the platforms that support video shorts

Video costs a lot of time and effort to make. At my current skill level, one video short takes about a day of work. But once you invest this cost, now your finished video can be thought of as capital that you own. You can show repeatedly, for little marginal cost, on all the platforms where people typically go to dump their garbage or to watch other people’s trash:

• YouTube
• Instagram
• Pinterest
• Tumblr
• TikTok
• Triller
• Snapchat
• Twitter
• LinkedIn

Only losers pay for video editing

Before you go and pirate Adobe Premier from a torrent site, or pay some dude on Fiverr to make you a video, consider this: You can do it with open source software for zero cost, other than your time. And it’s a useful skill that you want to learn yourself, not delegate to some marketing-agency-dwelling type of creep. (Sorry, sorry, I promised I’m done with hating on marketing. I will be on my best behavior from now on.)

I was under the impression that OpenShot was somewhat limited and buggy. That’s because I used to install it via the Ubuntu repositories on my machine. For some reason, the repository has an ancient 2.x version of OpenShot which is very buggy. It crashes often, and produces nasty sound artifacts. This is not the case with the latest version of OpenShot. You should take a minute to add their official PPA and install the latest stable version from there.

Learning OpenShot is a breeze. It’s popular, so there are countless tutorials on Youtube. Focus on learning how to do animations by adding keyframes to your media. I will discuss here only two technical aspects that I found required a little bit of digging: How to create video shorts and how to add subtitles.

Shorts custom video profile

When you start a video project, you want to define the output video format. There are several templates available, but none corresponds directly to the specifications that we want for a video short. We are going to create a custom profile. Create the following file: ~/.openshot_qt/profiles/shorts with the following content:

description=Short Vertical Video
frame_rate_num=24000
frame_rate_den=1000
width=1080
height=1920
progressive=1
sample_aspect_num=1
sample_aspect_den=1
display_aspect_num=9
display_aspect_den=16

If you’re wondering what these mean, you can have a look at the documentation for custom profiles.

Now you can start a new project with the “Short Vertical Video” profile.

Creating subs, the DIY way

There are, of course, a multitude of speech-to-text AI tools that will create subtitles for you automatically. One that works well is SubMagic. It offers many features including colors and emojis, and is well suited for social media videos.

But if your video is not too long, you can do it all by yourself. The advantage is that you don’t pay any money, and you are certain that there are no mistakes. You also have full control on where to break your sentences.

The best way to manually create subs is, surprisingly, not using a tool such as AegisSub, although it too can work well. I’ve found that the best tool for the job is Audacity. Here’s how you do it:

1. First, render the audio of your final video.
2. Load your audio in Audacity.
3. Go to Tracks → Add New → Label Track.
4. Select the first word. You can press Shift+Space to listen to your selection on repeat and ensure that you’ve selected the whole word and nothing more.
5. Press Ctrl+B to add a label.
6. In the box, type the word. In CAPITAL letters. This is for the TikTok generation. Don’t use punctuation unless absolutely necessary.
7. Deselect your selection, and select the next word. Audacity will help you by snapping the beginning of your selection to the end of the label you just entered.
8. Repeat steps 4 to 7 until you have done the entire length of the track. It gets easier as you learn how each phoneme looks like as a wave on the screen. For a 30 to 45 second video, it’s very doable.
9. Save your project as an .aup file, because you may want to go back to it and make changes, especially if you made a mistake.
10. Go to File → Export → Export labels. Save your labels to a text file. This file is not yet in a format that can be imported into OpenShot.
11. Go to https://magcius.github.io/audaciter/ and use the tool to convert your labels file to a SubRip (.srt) file. SubRip is almost the format that we want to use in OpenShot, but not quite.
12. In SubRip, each entry starts off with a line that is the numerical index of the entry, then another line with the time range (two timestamps separated by -->), and finally one or more lines of text, followed by a blank line. OpenShot doesn’t like the line with the numerical index, so we remove it from our .srt file. You can replace the following regular expression that matches these lines, with the empty string: ^\d+$
13. Go to your video project in OpenShot. With a finished video, you will have several tracks full of small fragments of video clips, audio clips, and graphics being animated. You don’t want to attach your subs to any of that mess. Create a separate track for your subs.
14. In the new track, add a 1080×1920 PNG image that is completely transparent. Here’s one.
15. Stretch the image so it “appears” over the entire length of the video. (Of course, if your image is truly transparent, it won’t actually appear.)
16. Ensure that your cursor is positioned at the beginning of your track. (Use Ctrl+← to navigate to the beginning). If you don’t do this, when you later edit the Caption properties you will create a new keyframe, and you don’t want that. You want the properties that you’re about to enter to apply to the entire track.
17. Go to the Effects tab, and drag the Caption effect onto the transparent image in your subtitles track.
18. The letter C will appear on the track. Click on it to edit the properties of the Captions effect.
19. I will share here the settings that I like to use in my videos. You can experiment with other settings of course. Font: Arial Black Bold 100pt (or any heavy font). Font size: 65. Font alpha: 255. Font color: Yellow #FFFF00. Stroke width: 3. Border: black #000000. Left size: 0. Right size: 0. Top size: 0.75.
20. Now go to the right side of the screen, and paste the SubRip subs (with the index lines removed).
21. Watch the video to ensure that all the subs are shown. If a line is too wide, it will not be shown. So check the lines with the most text. If a line is not shown, either break it into two, or reduce the font size. Your lines should be individual words, so they should not be too long.

That’s it. You can now render your ad and spam-post it on all the aforementioned platforms. Add it into your articles, above the fold. Then, post links to your video shorts again and again. Repetition is key.

The post 🎞 Engaging video shorts with subtitles, the open source way appeared first on Alexandros Georgiou.

Features

• Two WordPress sites: https://www.example1.com and https://www.example2.com.
• Redirects from https://example1.com, http://example1.com, http://www.example1.com to https://example1.com (and the same for example2.com).
• Let’s encrypt certificates.
• WordPress debug logging with logrotate. I have ranted previously about why I think having debug logging turned on is important on live sites.
• Emails must work in WordPress.
• The containers must run as a service, so that they start with system start, and exit gracefully on system shutdown.
• Daily backups of all WordPress files and MySQL databases.
• Networks of the two sites should be isolated for security.

Shameless plug of my referral links

We start with a hosted server. This can be a dedicated server or a server slice. Hosting providers that I like are:

• DigitalOcean – Using this link you get a $200, 60-day credit to try their products. If you spend $25 after your credit expires, I will get $25 in credit.
• Hostinger – You don’t get anything with this link, except for a great hosting service. Again, I get a commission from this link if you stick with Hostinger for 45 days. Think of it as my reward for writing such a great article for you.

I have a Debian droplet on DigitalOcean with 2GB of RAM, but with some tweaking it’s possible to squeeze two low-traffic WordPress sites in 1GB, if you really need to keep the monthly costs down.

First let’s get the (DNS) record straight

The first order of business is to setup the DNS records. We’re going to need two A records to point to our server’s IP, and two CNAME records that will be wwww. aliases of the bare domain. Oh, and we’ll need some NS records to point to the domain name provider (in this case Digital Ocean).

I like to keep the TTL (Time-To-Live) values low until I’m finished with my setup. I’ve set everything to 1800 seconds which is half an hour. Once I’m sure that everything is OK, I can increase the values to something larger like 14400 (four hours).

ssh

We are going to need to be able to login to the server with a passwordless setup.

Login as root to the new server via the admin console.

Create a regular user with adduser:

adduser yourusername

Then add the user to sudoers with:

usermod -aG sudo yourusername

(Replace yourusername with your username.)

Once we are on our local machine, we check if we already have an ssh key with:

ls -al ~/.ssh/id_*.pub

If we don’t have any, we can generate one with:

ssh-keygen -t rsa -b 4096 -C "your_email@domain.com"

Once we are sure that there is a key, we upload it to the new server with:

ssh-copy-id yourusername@server_ip_address

(Again replace yourusername with your remote username, and server_ip_address with your ip address. You will need to enter the password you entered in adduser.)

Docker compose

First, let’s install docker on the server by following the installation instructions for Debian. I am not going to repeat the instructions here. If you have chosen a different distro, follow the respective instructions.

We are going to create a docker-compose.yml file. This file describes how the different docker containers are orchestrated.

We are going to need four containers:

• Two databases for the two sites.
• Two WordPress installations.

I’m first going to show some simple compose configs with the basics, then we are going to add the bells and whistles. Here goes:

Two databases, sitting in a server

version: "3.8"

name: droplet

networks:
    net1:
    net2:

volumes:
  db1volume:
  db2volume:

services:

  db1:
    image: mysql:8.2.0
    networks:
      - net1
    restart: unless-stopped
    expose:
      - "3306"
    volumes:
      - db1volume:/var/lib/mysql
    environment:
      MYSQL_ROOT_PASSWORD: wp1_root_pass
      MYSQL_DATABASE: wp_db1
      MYSQL_USER: db1_user
      MYSQL_PASSWORD: db1_pass

  db2:
    image: mysql:8.2.0
    networks:
      - net2
    restart: unless-stopped
    expose:
      - "3306"
    volumes:
      - db2volume:/var/lib/mysql
    environment:
      MYSQL_ROOT_PASSWORD: wp2_root_pass
      MYSQL_DATABASE: wp_db2
      MYSQL_USER: db2_user
      MYSQL_PASSWORD: db2_pass

There’s already a lot going on here:

• We are defining our composition to have a name. Here I am using droplet. This will also be the prefix for the names of all the containers.
• We are defining two networks, net1 and net2. Only containers on the same network can talk to each other. We don’t want our example1.com WordPress to have any access to the MySQL database of example2.com.
• Next we are defining two identical mysql:8.2.0 containers, named db1 and db2.
• Each of the two databases is put in its respective network (net1 and net2).
• We want a database that has crashed to restart, unless we explicitly stop it.
• We are going to let the databases listen to TCP port 3306. This is the port where WordPress will connect. All other ports are firewalled.
• We are going to mount the /var/lib/mysql directories into docker volumes named db1volume and db2volume.
• Next we are going to use some environment variables that the startup script inside the mysql image recognizes. These will set up a root password, a new empty database, and a username/password pair that WordPress will use to access this new database. The startup script will do all the CREATE DATABASE, CREATE USER and GRANT magic for us. You can learn more about the MySQL docker image here.

A tale of two WordPresses

Next, let’s also add the two WordPress services (these also go under the services section along with the databases):

  wp1:
    image: wordpress:latest
    networks:
      - net1
    depends_on:
      - db1
    user: 1000:1000
    restart: unless-stopped
    expose:
      - "80"
    volumes:
      - ./wp1fs:/var/www/html
    ports:
      - "127.0.0.1:8101:80"
    environment:
      WORDPRESS_DB_HOST: db1:3306
      WORDPRESS_DB_NAME: wp_db1
      WORDPRESS_DB_USER: db1_user
      WORDPRESS_DB_PASSWORD: db1_pass
      WORDPRESS_DEBUG: true

  wp2:
    image: wordpress:latest
    networks:
      - net2
    depends_on:
      - db1
    restart: unless-stopped
    expose:
      - "80"
    volumes:
      - ./wp2fs:/var/www/html
    ports:
      - "127.0.0.1:8102:80"
    environment:
      WORDPRESS_DB_HOST: db2:3306
      WORDPRESS_DB_NAME: wp_db2
      WORDPRESS_DB_USER: db2_user
      WORDPRESS_DB_PASSWORD: db2_pass
      WORDPRESS_DEBUG: true

• We have named the two WordPress containers wp1 and wp2 and assigned them to our two networks, net1 and net2.
• We have defined that these depend on their respective databases to function.
• We have defined that these containers are to be restarted if they crash, but not if we explicitly stop them.
• We are exposing only HTTP port 80 to the networks. All other ports are firewalled. We are not exposing port 443 here. TLS encryption will be done at the host level that will run the reverse proxy (see below).
• We are mounting two local directories here ./wp1fs and ./wp2fs. These will contain the WordPress installations. The first time that the containers run, WordPress will be installed in them. A special wp-config.php file will be placed in there. This file pulls the DB connection settings from the environment variables that we specify below.
• We are port-mapping the HTTP 80 ports to the host’s ports 8101 and 8102. These are the ports that the reverse proxy will use. They are bound to the loopback network (127.0.0.1), and are therefore not exposed to the outside world. If we had used just 8101:80, this would map port 80 of the container to port 8101 of the host on all network interfaces, including the one facing the outside world. This is not ideal. We only want access to our services through our reverse proxy.
• The WORDPRESS_* environment variables are specific to this wordpress image. We specify the databases, the login credentials that we also specified above, and we turn on debug logging. To learn more about these environment variables, click here.

NOTE: I have made the decision here to put the databases into system volumes (these live usually in /var/lib/docker/volumes and can be shared between containers, the WordPress filesystems are mounted in local directories which I call wp1volume and wp2volume. If you prefer to have all volumes unde /var/lib, you can delete the ./ prefix in front of the volume names.

The bells and whistles

If you thought that’s enough, you are gravely mistaken. Here’s a few more things to take care of:

Database collation

We are going to set the databases a UTF-8 multibyte collation for unicode support. Under the environment variables in the database services, we are going to add an explicit mysqld command:

command: "mysqld --character-set-server=utf8mb4 --collation-server=utf8mb4_unicode_ci

And under the WordPress services, we are going to add the following environment variable:

  WORDPRESS_DB_COLLATE: utf8mb4_unicode_ci

File permissions

If we run the above containers, WordPress won’t be able to install or remove any themes or plugins, and it won’t be able to do anything that requires writing to the file system.

This is because, in the WordPress images, the user that runs apache has a different uid and guid than the file system. The files are owned by uid 1000 and guid 1000. We can specify that the user running stuff inside the container has the same numeric ids. To do this, we add the following to the two WordPress services:

user: 1000:1000

Database memory

By default, a mysql instance will take up at least 360MB of memory once it’s running. Most of it is because of the Performance Schema instruments, which take up a lot of memory.

The Performance Schema is a database that keeps track of the mysqld server’s performance, and is useful for diagnostics. If you are not going to use this feature, then you can turn it off. The memory usage of each DB container will then fall to a little over 100MB.

We are going to create a file named disable-perf-schema.cnf with the following contents:

[mysqld]
performance_schema = OFF

This will be added to the mysql server’s config files. The server includes any .cnf files in the /etc/mysql/conf.d directory into its configuration. We can use the volumes section to map this file into our two db containers:

volumes:
  - db1:/var/lib/mysql
  - ./disable-perf-schema.cnf:/etc/mysql/conf.d/disable-perf-schema.cnf

volumes:
  - db2:/var/lib/mysql
  - ./disable-perf-schema.cnf:/etc/mysql/conf.d/disable-perf-schema.cnf

There are more hacks to reduce the memory usage of mysqld, but these are beyond the scope of this article. For example, you can look into reducing the InnoDB buffer pool size.

Log rotate

We have enabled debug logging, because reasons. This is cool, but the /var/www/html/wp-content/debug.log files will eventually fill up our containers if left unchecked. Enter logrotate to the rescue:

We are going to create a file named wordpress.logrotate with the following content:

/var/www/html/wp-content/debug.log
{
        su 1000 1000
        rotate 24
        copytruncate
        weekly
        missingok
        notifempty
        compress
}

This will gzip old logs daily and will delete even older logs. If you are not sure about the details, ChatGPT and Bard can explain exactly what each line does.

Note how we use again the uid and guid of the WordPress image.

Let’s mount this file into our WordPress containers, by adding a line to their volume clause:

volumes:
  - ./wp1fs:/var/www/html
  - ./wordpress.logrotate:/etc/logrotate.d/wordpress

volumes:
  - ./wp2fs:/var/www/html
  - ./wordpress.logrotate:/etc/logrotate.d/wordpress

Docker compose recap

We now have the following docker-compose.yml file:

version: "3.8"

name: droplet

networks:
    net1:
    net2:

volumes:
  db1volume:
  db2volume:

services:

  db1:
    image: mysql:8.2.0
    networks:
      - net1
    restart: unless-stopped
    expose:
      - "3306"
    volumes:
      - db1volume:/var/lib/mysql
      - ./disable-perf-schema.cnf:/etc/mysql/conf.d/disable-perf-schema.cnf
    environment:
      MYSQL_ROOT_PASSWORD: wp1_root_pass
      MYSQL_DATABASE: wp_db1
      MYSQL_USER: db1_user
      MYSQL_PASSWORD: db1_pass
    command: "mysqld --character-set-server=utf8mb4 --collation-server=utf8mb4_unicode_ci --performance-schema-instrument='%=OFF' --innodb-buffer-pool-size=32M"

  db2:
    image: mysql:8.2.0
    networks:
      - net2
    restart: unless-stopped
    expose:
      - "3306"
    volumes:
      - db2volume:/var/lib/mysql
      - ./disable-perf-schema.cnf:/etc/mysql/conf.d/disable-perf-schema.cnf
    environment:
      MYSQL_ROOT_PASSWORD: wp2_root_pass
      MYSQL_DATABASE: wp_db2
      MYSQL_USER: db2_user
      MYSQL_PASSWORD: db2_pass
    command: "mysqld --character-set-server=utf8mb4 --collation-server=utf8mb4_unicode_ci --performance-schema-instrument='%=OFF' --innodb-buffer-pool-size=32M"

  wp1:
    image: wordpress:latest
    networks:
      - net1
    depends_on:
      - db1
    user: 1000:1000
    restart: unless-stopped
    expose:
      - "80"
    volumes:
      - ./wp1fs:/var/www/html
      - ./wordpress.logrotate:/etc/logrotate.d/wordpress
    ports:
      - "8101:80"
    environment:
      WORDPRESS_DB_HOST: db1:3306
      WORDPRESS_DB_NAME: wp_db1
      WORDPRESS_DB_USER: db1_user
      WORDPRESS_DB_PASSWORD: db1_pass
      WORDPRESS_DB_COLLATE: utf8mb4_unicode_ci
      WORDPRESS_DEBUG: true

  wp2:
    image: wordpress:latest
    networks:
      - net2
    depends_on:
      - db1
    user: 1000:1000
    restart: unless-stopped
    expose:
      - "80"
    volumes:
      - ./wp2fs:/var/www/html
      - ./wordpress.logrotate:/etc/logrotate.d/wordpress
    ports:
      - "8102:80"
    environment:
      WORDPRESS_DB_HOST: db2:3306
      WORDPRESS_DB_NAME: wp_db2
      WORDPRESS_DB_USER: db2_user
      WORDPRESS_DB_PASSWORD: db2_pass
      WORDPRESS_DB_COLLATE: utf8mb4_unicode_ci
      WORDPRESS_DEBUG: true

We can start this with docker compose up (we must first cd into the same directory as the .yml file).

We can see if it’s running with docker compose ls, and we can see the containers with docker container ls.

We can inspect memory usage with docker stats.

We can stop the containers with docker compose down.

If we also want to wipe the database volumes and start over, we can do docker compose down -v (DESTRUCTIVE!!!).

We can go into the shell of the first database with:

docker exec -it droplet-db1-1 bash

And then, we can go into the mysql console with

mysql -u root -pwp1_root_pass

We can go into the shell of the first WordPress with:

docker exec -it droplet-wp1-1 bash

If we need to, we can install wp-cli using instructions from https://wp-cli.org/. The copy of wp-cli will not be persisted into the container across restarts. (Note: it’s possible to add special containers with wp-cli pre-installed, but again this is out of scope of this article. For more information, see the CLI images here.

DaaS (Docker-as-a-Service)

We don’t want to have to issue docker compose up every time the server starts, and docker compose down every time the server stops. Let’s create a systemd unit, so that it runs as a service.

We’ll create a file named /etc/systemd/system/docker-compose.service with the following carefully crafted contents:

[Unit]
Description=A bunch of containers
After=docker.service
Requires=docker.service

[Service]
Type=oneshot
RemainAfterExit=yes
User=yourusername
ExecStart=/bin/bash -c "docker compose -f /home/yourusername/docker-compose.yml up --detach"
ExecStop=/bin/bash -c "docker compose -f /home/yourusername/docker-compose.yml stop"

[Install]
WantedBy=multi-user.target

• Replace yourusername with your username (duh!).
• Replace the description with something less silly (optional).
• Note how we only start this service after the docker service starts.
• Note that we do a --detach. This will start the containers in the background and exit, without showing the logs of all the containers in the standard output.

We can now start the service with

sudo service docker-compose up

And stop it with

sudo service docker-compose down

If we want to see the logs of all the containers, we can type

docker compose logs -f

We should now be able to do curl http://127.0.0.1:8101 and see the HTML of the front page of the first WordPress.

The reverse proxy

The database and WordPress containers are running, but they are not yet exposed to the outside world. To do this, we are going to use nginx as a reverse proxy.

The reverse proxy will:

• handle all the redirects that we need
• expose the apache2 servers to the outside world
• handle the TLS encryption

First we setup Let’s Encrypt. How to do this is beyond the scope of this article. You can look here for a good introduction.

The bottom line is that certbot must be installed, and the following public and private certificate files must exist on your server (host):

/etc/letsencrypt/live/example1.com/fullchain.pem
/etc/letsencrypt/live/example1.com/privkey.pem
/etc/letsencrypt/live/example2.com/fullchain.pem
/etc/letsencrypt/live/example2.com/privkey.pem

These files are actually symlinks to the latest certificate issued. This is all handled by certbot.

Let’s start to create an nginx config file, which we will place in /etc/nginx/sites-available/reverse-proxy.conf.

We are going to enter several server stanzas, remembering that nginx will use the first one that matches in order from top to bottom.

Redirects from http to https

First, we want any unencrypted requests to port 80 to do a soft redirect to our https://www. sites.

server {
    listen       80;
    listen       [::]:80;
    server_name example1.com;
    return 302 https://www.example1.com$request_uri;
}

server {
    listen       80;
    listen       [::]:80;
    server_name example2.com;
    return 302 https://www.example2.com$request_uri;
}

The first listen statement is for IPv4, and the second is for IPv6. We redirect to the TLS site, preserving the path segment of the request URI.

Proxy forwarding

Next we are going to enter the stanza that handles the actual site content:

server {
    listen      443 ssl;
    listen      [::]:443 ssl;
    server_name www.example1.com;

    ssl_certificate /etc/letsencrypt/live/example1.com/fullchain.pem;
    ssl_certificate_key /etc/letsencrypt/live/example1.com/privkey.pem;
    include /etc/letsencrypt/options-ssl-nginx.conf;

    location / {
        proxy_pass http://127.0.0.1:8101/;
        proxy_set_header Host $host;
        proxy_set_header X-Real-IP $remote_addr;
        proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
        proxy_set_header X-Forwarded-Proto $scheme;
    }
}

server {
    listen      443 ssl;
    listen      [::]:443 ssl;
    server_name www.example2.com;

    ssl_certificate /etc/letsencrypt/live/example2.com/fullchain.pem;
    ssl_certificate_key /etc/letsencrypt/live/example2.com/privkey.pem;
    include /etc/letsencrypt/options-ssl-nginx.conf;

    location / {
        proxy_pass http://127.0.0.1:8102/;
        proxy_set_header Host $host;
        proxy_set_header X-Real-IP $remote_addr;
        proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
        proxy_set_header X-Forwarded-Proto $scheme;
    }
}

Again, we are listening for 443 (the TLS port) on both IPv4 and IPv6.

Notice how we only listen for requests to the www. subdomain here.

We use the TLS certificates first, then we specify the reverse proxy in the location / section.

We forward each site to the correct port that we exposed with docker (8101 and 8102 in this case).

We also set some X- headers. This is so that the PHP server knows some details about the client.

Redirects from all subdomains to www

Finally, we want requests from https://example1.com, or from ay other subdomain, such as https://foo.example1.com, to redirect to our www. subdomain:

server {
    listen 443 ssl;
    listen [::]:443 ssl;
    server_name .example1.com;

    ssl_certificate /etc/letsencrypt/live/example1.com/fullchain.pem;
    ssl_certificate_key /etc/letsencrypt/live/example1.com/privkey.pem;
    include /etc/letsencrypt/options-ssl-nginx.conf;

    location / {
        rewrite ^ https://www.example1.com permanent;
    }
}

server {
    listen 443 ssl;
    listen [::]:443 ssl;
    server_name .example2.com;

    ssl_certificate /etc/letsencrypt/live/example2.com/fullchain.pem;
    ssl_certificate_key /etc/letsencrypt/live/example2.com/privkey.pem;
    include /etc/letsencrypt/options-ssl-nginx.conf;

    location / {
        rewrite ^ https://www.example2.com permanent;
    }
}

Here we listen for any subdomain. Note the dot (.) prefix in the server_name.

We again use the TLS certificates, but this time we perform a redirect to the wwww. subdomain.

Administering our reverse proxy

When we are ready to enable our reverse proxy, we will create a symlink to sites-enabled:

sudo ln -s /etc/nginx/sites-available/reverse-proxy.conf /etc/nginx/sites-enabled/reverse-proxy.conf

We can test our syntax to see that it is correct with:

sudo nginx -t

And finally we can restart the nginx server with:

sudo service nginx restart

We can check the status of the server with:

sudo service nginx status

If everything is working correctly, and if the DNS records have had time to propagate, then we can visit our sites and run the famous WordPress installation process:

• https://www.example1.com/
• https://www.example2.com/

Emails

If only the above was enough. Sadly, our WordPress installations need a way to send emails, otherwise the webmaster experience is going to suck big time.

I say sadly, because setting up sendmail first on the host is relatively easy, but then setting up SMTP proxies in the WordPress containers is not something I am familiar with. Sorry guys, in the interest of keeping things simple, I’m going to cheat a little here. Here’s what I did:

• Install the free WP Mail SMTP plugin on both sites.
• Create an application-specific password in my google account.
• In the WordPress admin screens, go to: WP Mail SMTP → Mailer → Other SMTP.
• Enter the following settings:
  • SMTP Host: smtp.gmail.com
  • Encryption: SSL
  • SMTP Port: 465
  • Auto TLS: ON
  • Authentication: ON
  • SMTP Username: (my gmail address)
  • SMTP Password: (the application specific password that I just created).
• Hit Save Settings.
• Go to WP Mail SMTP → Tools and send a test email.

If everything works, then WordPress and its plugins can now send emails. But it will only be able to send email into spam folders, until we add a Sender Policy Framework (SPF) record to our DNS entries:

The above TXT records tell recipients to treat all emails coming from servers pointed to by the A or MX record of your domains as safe, and others as potentially suspicious. Again, use your favorite AI chatbot to constuct an SPF record that matches your needs.

Nothing more permanent than a 301 redirect

If all works, it’s now time to turn the soft redirects into permanent (hard) redirects. Edit the reverse proxy config and change any 302 redirects to 301. Any browsers visiting your site will cache these redirects for eternity.

It’s also now a good time to increase the Time-to-Live of all the DNS records to something like 4 hours, or 14400 seconds.

Backups

You would think that by now you’re finished, but you’d be wrong!

Any IT technician worth their salt knows that they must backup, and backup often.

First, turn off the server or droplet and take a full backup, snapshot, or whatever. Future you will thank you.

Then, let’s see how we can take automated daily backups. We can either pay the hosting provider every month to do this for us, or we can spend a few minutes to set up a few cron jobs. Let’s be cheap and do it manually.

I have a raspberry Pi at home that is always on. It does various things like take backups, ping various services and email me if they are down, trigger wp-cron URLs, control crypto miners, run services I need such as my ticket system, and in general runs any other odd 24/7 task. You should also have one such low-power system. The great thing with Raspberry Pi is that it’s easy to take out the MicroSD and gzip it into a mechanical disk, so the backup mechanism itself is nicely backed up in its entirety. (Yo dawg, heard you like backups…)

We’ll now use our local always-on Linux system to take daily backups of our online filesystems and databases:

Local backups.sh script

First, let’s create a DB user that only has enough access to take backups from both databases, but no more:

Login to the MySQL consoles of each database and create a wp_bu user that will do backups:

CREATE USER 'wp_bu'@'localhost' IDENTIFIED BY 'SOMESTRONGPASSWORD';
GRANT SELECT, LOCK TABLES ON wp_db1.* TO 'wp_bu'@'localhost';

CREATE USER 'wp_bu'@'localhost' IDENTIFIED BY 'SOMESTRONGPASSWORD';
GRANT SELECT, LOCK TABLES ON wp_db2.* TO 'wp_bu'@'localhost';

We only need SELECT, but since we want to call mysqldump with the --single-transaction argument, we’ll also need to grant the LOCK TABLES permission. No point in having an ACID database if we’re going to take backups of an inconsistent state now, is there?

We’ll now create a bash shell script that does our daily backups. Let’s place it in our local backup server and call it backups.sh:

#!/bin/bash

# ensure dirs exist
mkdir -p /path-to-backups/cache/wp{1,2}volume /path-to-backups/server

# download DBs to SQL files
ssh -t server "docker exec droplet-wpdb-1 nice -n 19 mysqldump -u wp_bu -pSOMESTRONGPASSWORD --no-tablespaces --single-transaction wp_db1 | nice -n 19 gzip -9 -f" >/path-to-backups/server/wp_db1-`date --rfc-3339=date`.sql.gz
ssh -t server "docker exec droplet-wpdb-2 nice -n 19 mysqldump -u wp_bu -pSOMESTRONGPASSWORD --no-tablespaces  --single-transaction wp_db2 | nice -n 19 gzip -9 -f" >/path-to-backups/server/wp_db2-`date --rfc-3339=date`.sql.gz

# download wp-content files to backup cache
rsync -aq server:~/wp1fs/* /path-to-backups/cache/wp1volume
rsync -aq server:~/wp2fs/* /path-to-backups/cache/wp2volume

# Zip downloaded wp-content files
zip -r9q /path-to-backups/server/wp1-`date --rfc-3339=date`.zip /path-to-backups/cache/wp1volume -x "**/GeoLite2*" -x "**/GeoIPv6.dat"
zip -r9q /path-to-backups/server/wp2-`date --rfc-3339=date`.zip /path-to-backups/cache/wp2volume -x "**/GeoLite2*" -x "**/GeoIPv6.dat"

# prune old DB and FILE backups from local backups
cd /path-to-backups/server && ls -1tr | head -n -30 | xargs -d '\n' rm -rf -

Again, a lot goes on here. Let’s unpack:

• The script creates directories server and cache under /path-to-backups. Replace this path with something that points to the directory where you want to keep your backups.
• We then ssh to the host using the -t argument because we are in a headless environment (cron). We issue a docker exec command into our databases. Notice how we do not use the -it arguments to docker exec, since this is a headless command (no TTY attached). The command is a mysqldump command that uses the credentials we just created to export the databases in a single transaction each. The SQL output is compressed with maximum compression (-9) and the binary output of gzip is forced (-f) into the standard output, which is then sent over the ssh connection. In our local backups server, we redirect this compressed stream into an .sql.gz file. The file name starts with wp_db1- and includes the current date in YYYY-MM-DD notation. (RFC 3339 is my idea of a perfect date, btw). The --no-tablespaces argument is need in MySQL 8.0.21 and later, otherwise you’ll need the PROCESS global permission. (Unless you are using tablespaces you don’t need it, hence the argument --no-tablespaces.) Notice that we make sure to be nice to other running processes because we don’t want to impact the performance of the web server with our backups. 19 is the idle CPU priority.
• We then use rsync with the quiet (-q) and archive (-a) flags to copy the files of our WordPress installations into our cache/wp1volume and cache/wp2volume directories. The advantage of using rsync is that only changes to these directories will be transferred.
• We then create a zip file for each of these directories. We name the zip files with the prefixes wp1- and wp2- followed again by our idea of a perfect date. Many WordPress plugins include a database of IPs mapped to geographical locations. These files are large and can be found online. If we don’t want to save these, we can exclude them (-x flag), but this is optional.
• Finally we list the files we created (both .sql.gz and .zip files) and we only keep the last 30, deleting any older ones. Since we have two files for each of two databases, this will retain daily backups for the last week or so.

Make the script executable with

chmod +x backups.sh

We run the script once, and we check the .sql.gz files using zless and the zip files with unzip -l.

Once we are certain that all data is backed up by the script, we add it to the crontab. Edit the crontab with crontab -e and add the line:

20 4 * * * /bin/bash /home/yourusername/backups.sh

This will execute the backups every day at 4:20 in the morning.

Checking the backups

The server works and is fully backed up. You would think that you’re done by now. That’s where you’d be wrong again!

Having backups and not checking them regularly is worse than not having backups at all: You are being lulled into a false sense of security. You may act precariously, thinking that you can always go back to the last backup. However, all backup mechanisms can fail, for any number of reasons.

What I do, is I’ve set up a weekly reminder in my Google calendar to check the backups. It only takes half a minute per week to ssh into my backup server and do an ls -l, thus ensuring that the latest backups exist, and their file size is what I’d expect. I keep old backups for about a week, hence the weekly reminder.

I also have another reminder every three months, to backup the MicroSD of my Raspberry Pi backup server. Once every three months, I shutdown the Pi, take out the MicroSD, put it into my work PC, and copy the entire image into a file, stored on my mechanical disk:

sudo dd if=/dev/sdf of=/mnt/bu/rpi-backup-`date --iso-8601=date`.img bs=4096 conv=sync,noerror status=progress
gzip -9 /mnt/bu/rpi-backup-`date --iso-8601=date`.img

Only once I have this process setup I can sleep at night.

Are we finished yet?

By now you would think that we’re not finished yet, and that there’s more things to do. That’s where you’d be wrong!

And for anyone wondering, example1.com is actually https://www.dashed-slug.net and example2.com is actually this blog, https://www.alexgeorgiou.gr. There’s also a plain nginx container in there that serves static HTML files at https://wallets-phpdoc.dashed-slug.net .

My config is actually a little bit more complex than the one discussed above. To save some more server memory, I had to put both databases into the same MySQL container, and set up two different DB users with access restricted to each respective database. But you shouldn’t do this at home, because isolation!

This article is being served by the containers I discussed here, and will be backed up early tomorrow morning, via the mechanism I shared with you above. Which is pretty meta, if you think about it!

I never expected to compose such a long, self-contained article on containers and docker compose. But now it’s finished and I can hardly contain my excitement!

Thanks for sticking to the end. Hope you enjoyed.

The post 📦 Two dockerized WordPress sites, with Let’s Encrypt, logging, SMTP relay, controlled by a systemd service, and daily backups appeared first on Alexandros Georgiou.

Like any self-respecting server-side app, WordPress also features logging. While this is usually discouraged, themes and plugins can write to this log with error_log. Any PHP warnings or errors will also be dumped to the logs. That is, if logging is enabled.

How to enable logging: RTFM

To enable logging, edit the wp-config.php file.

First, add define( 'WP_DEBUG', true );. This will enable “debug” logging, as explained in the article Debugging in WordPress.

Then, add define( 'WP_DEBUG_LOG', true ); to make the log be written to a file, /wp-content/debug.log.

Finally, you don’t want any PHP warnings or errors to spill over into your site’s markup and cause problems. Add define( 'WP_DEBUG_DISPLAY', false ); and @ini_set( 'display_errors', 0 ); for good measure.

Great! Now you’re good to go. Debug logging has been enabled.

“Debug” logging?

But I take some issue with that. Why is logging labelled “debug” logging? Are there no other legitimate reasons to keep logs? WordPress documentation doesn’t think so:

It is not recommended to use WP_DEBUG or the other debug tools on live sites; they are meant for local testing and staging installs.

from: Debugging in WordPress

Perhaps this is OK for most people. WordPress is meant to be used by non-programmers, and generally by people who wouldn’t know what to do with a log file. But if you can jam with the console cowboys in cyberspace, I think you should keep logging on, even in production. Here’s why:

Health status

The WordPress ecosystem is a beautiful mess. Typically you will run all kinds of plugins of varying quality. I know that I am responsible for dumping some low-effort buggy code on to the WordPress.org repository, and, believe me, I am not the only one. When you put many pieces of software together in the same system, things can break. Yes, you have the PHP error logs which capture most errors, but it’s helpful to have a record of the errors and any error_log output. If you have live logs, you can address incidents faster. In my mind it’s a no-brainer.

Security

In case of someone breaking in to your site, and yes, this happens a lot, it may be possible to figure out what happened by looking at the logs. You may be able to figure out which plugin the hacker exploited and in what way, by looking at the failed attempts that preceded the successful break-in.

For extra credit, set up a cron job that stores backups off-site to another server. Evenif a hacker manages to clear the logs, you still have data for your forensics investigation.

Disk space

The main arguments against logging are that

1. writing logs degrades performance, and
2. writing logs can fill up disk space

Argument #1 is not even worth discussing in the year of our Lord 2023, when SSDs are the norm on server hardware. I can guarantee you that even if you care about performance a lot, the milliseconds wasted on logging are the least of your concerns. I’d much rather have logs available when I need them, if this is going to save me time and money by avoiding or fixing incidents.

Argument #2 is perhaps less laughable, but still invalid. The disk space argument is only brought up by people who haven’t heard of logrotate.

Let me show you my /etc/logrotate.d/wordpress file on this blog’s server:

/usr/share/nginx/alexg/wp-content/debug.log
{
        su www-data www-data
        rotate 24
        copytruncate
        weekly
        missingok
        notifempty
        compress
}

And now let me show you the log files that this generates:

$ ls -l /usr/share/nginx/alexg/wp-content/debug*
-rw-r--r-- 1 www-data www-data 105705 Mar 12 01:43 /usr/share/nginx/alexg/wp-content/debug.log
-rw-r--r-- 1 www-data www-data   3928 Mar  5 06:34 /usr/share/nginx/alexg/wp-content/debug.log.1.gz
-rw-r--r-- 1 www-data www-data   3181 Jan  2 06:09 /usr/share/nginx/alexg/wp-content/debug.log.10.gz
-rw-r--r-- 1 www-data www-data   2532 Dec 25 06:36 /usr/share/nginx/alexg/wp-content/debug.log.11.gz
-rw-r--r-- 1 www-data www-data   2991 Dec 19 04:44 /usr/share/nginx/alexg/wp-content/debug.log.12.gz
-rw-r--r-- 1 www-data www-data   3256 Dec 11 03:15 /usr/share/nginx/alexg/wp-content/debug.log.13.gz
-rw-r--r-- 1 www-data www-data   4397 Dec  5 05:30 /usr/share/nginx/alexg/wp-content/debug.log.14.gz
-rw-r--r-- 1 www-data www-data   2284 Nov 27 06:15 /usr/share/nginx/alexg/wp-content/debug.log.15.gz
-rw-r--r-- 1 www-data www-data   2866 Nov 20 06:24 /usr/share/nginx/alexg/wp-content/debug.log.16.gz
-rw-r--r-- 1 www-data www-data   3339 Nov 13 04:22 /usr/share/nginx/alexg/wp-content/debug.log.17.gz
-rw-r--r-- 1 www-data www-data  15378 Nov  6 06:20 /usr/share/nginx/alexg/wp-content/debug.log.18.gz
-rw-r--r-- 1 www-data www-data   3194 Oct 30 06:18 /usr/share/nginx/alexg/wp-content/debug.log.19.gz
-rw-r--r-- 1 www-data www-data  38415 Feb 27 06:30 /usr/share/nginx/alexg/wp-content/debug.log.2.gz
-rw-r--r-- 1 www-data www-data   3619 Oct 24 06:24 /usr/share/nginx/alexg/wp-content/debug.log.20.gz
-rw-r--r-- 1 www-data www-data   3827 Oct 16 05:11 /usr/share/nginx/alexg/wp-content/debug.log.21.gz
-rw-r--r-- 1 www-data www-data   2213 Oct  9 05:52 /usr/share/nginx/alexg/wp-content/debug.log.22.gz
-rw-r--r-- 1 www-data www-data   3332 Oct  3 06:05 /usr/share/nginx/alexg/wp-content/debug.log.23.gz
-rw-r--r-- 1 www-data www-data  13983 Sep 25 06:24 /usr/share/nginx/alexg/wp-content/debug.log.24.gz
-rw-r--r-- 1 www-data www-data   2863 Feb 19 05:09 /usr/share/nginx/alexg/wp-content/debug.log.3.gz
-rw-r--r-- 1 www-data www-data   3913 Feb 13 05:45 /usr/share/nginx/alexg/wp-content/debug.log.4.gz
-rw-r--r-- 1 www-data www-data   3276 Feb  5 06:00 /usr/share/nginx/alexg/wp-content/debug.log.5.gz
-rw-r--r-- 1 www-data www-data   3354 Jan 29 05:38 /usr/share/nginx/alexg/wp-content/debug.log.6.gz
-rw-r--r-- 1 www-data www-data   4171 Jan 22 06:12 /usr/share/nginx/alexg/wp-content/debug.log.7.gz
-rw-r--r-- 1 www-data www-data   2835 Jan 15 06:21 /usr/share/nginx/alexg/wp-content/debug.log.8.gz
-rw-r--r-- 1 www-data www-data   2694 Jan  8 05:04 /usr/share/nginx/alexg/wp-content/debug.log.9.gz

If you’re wondering how much space this takes:

$ du -ch /usr/share/nginx/alexg/wp-content/debug*
108K    /usr/share/nginx/alexg/wp-content/debug.log
4.0K    /usr/share/nginx/alexg/wp-content/debug.log.10.gz
4.0K    /usr/share/nginx/alexg/wp-content/debug.log.11.gz
4.0K    /usr/share/nginx/alexg/wp-content/debug.log.12.gz
4.0K    /usr/share/nginx/alexg/wp-content/debug.log.13.gz
8.0K    /usr/share/nginx/alexg/wp-content/debug.log.14.gz
4.0K    /usr/share/nginx/alexg/wp-content/debug.log.15.gz
4.0K    /usr/share/nginx/alexg/wp-content/debug.log.16.gz
4.0K    /usr/share/nginx/alexg/wp-content/debug.log.17.gz
16K /usr/share/nginx/alexg/wp-content/debug.log.18.gz
4.0K    /usr/share/nginx/alexg/wp-content/debug.log.19.gz
4.0K    /usr/share/nginx/alexg/wp-content/debug.log.1.gz
4.0K    /usr/share/nginx/alexg/wp-content/debug.log.20.gz
4.0K    /usr/share/nginx/alexg/wp-content/debug.log.21.gz
4.0K    /usr/share/nginx/alexg/wp-content/debug.log.22.gz
4.0K    /usr/share/nginx/alexg/wp-content/debug.log.23.gz
16K /usr/share/nginx/alexg/wp-content/debug.log.24.gz
40K /usr/share/nginx/alexg/wp-content/debug.log.2.gz
4.0K    /usr/share/nginx/alexg/wp-content/debug.log.3.gz
4.0K    /usr/share/nginx/alexg/wp-content/debug.log.4.gz
4.0K    /usr/share/nginx/alexg/wp-content/debug.log.5.gz
4.0K    /usr/share/nginx/alexg/wp-content/debug.log.6.gz
8.0K    /usr/share/nginx/alexg/wp-content/debug.log.7.gz
4.0K    /usr/share/nginx/alexg/wp-content/debug.log.8.gz
4.0K    /usr/share/nginx/alexg/wp-content/debug.log.9.gz
272K    total

So, full logs for the last 24 days are taking up 272 kilobytes when compressed. Disk space well spent, if you ask me.

wp-console plugin

Even a console cowboy will agree that sometimes it’s more convenient to view the logs via the WordPress admin. For these rare occasions, I use wp-console. Next to the Console tab, which is very useful by itself, there is the Debug Log tab.

Make a habbit out of it

Let’s take a step back: WordPress is usually the main engine of a small business. Running a business means executing a set of processes repeatedly, with the hope that these processes will create value.

I like to be proactive: One of the processes that I execute repeatedly as part of my business, is to visit the site logs every 3 months. I have a calendar reminder for this, pointing to a ticket that’s all about inspecting logs. As I identify problems that I want to fix, I open separate tickets for those as well.

Logs are compressed

First, I look at the logs. Today’s file is uncompressed (debug.log), while older files are gzipped (debug.*.log.gz). Instead of ungzipping the files, I usually like to inspect them in place via an ssh shell, using the zTools:

• use zcat instead of cat
• use zless instead of less
• use zgrep instead of grep
• use zfgrep instead of fgrep
• use zegrep instead of egrep

How I inspect logs

Usually, when you look at such logs, you will see one single type of warning or notice being repeated a lot, possibly because of a plugin or theme.

Let’s say you see a “PHP Notice: Undefined variable: foo” line a lot, coming from some plugin. It’s not causing any real problems, but it gets written out with every HTTP request. You are not responsible for maintaining this plugin, but this line is obscuring your view of the logs. So, just look at the logs without it:

zfgrep -v 'PHP Notice:  Undefined variable: foo' debug.log.*.gz

Now you will see a listing of all your logs, except for that notice. The next most common notice will become apparent. Let’s say that this is a “PHP Notice: Undefined index: bar” in another plugin, that you are also not the maintainer of. You can filter for that as well. Just daisy-chain your greps using pipes:

zfgrep -v 'PHP Notice:  Undefined variable: foo' debug.log.*.gz | fgrep -v 'PHP Notice:  Undefined index: bar'

You could go into the source code of these files and add a @ character in front that line, to suppress the warnings, at least until the plugin in question is updated by the developer. But I don’t recommend it. Too much effort for too little gain. gzip does an excellent job of compressing similar strings repeated thousands of times. This is literally what huffman trees were built for.

Rinse and repeat

Continue chaining greps, until you get to an error that you care about. Maybe it’s an error that looks ugly, worries you, or just feels bad. Maybe an include is missing. Maybe you can investigate, or even fix the issue. Or maybe you feel like you have to notify the developer via the plugin’s support forum. Maybe you’ll decide that a plugin causes too many errors and is easily replaceable by another similar one.

Deprecated code

Depending on what you are doing, you may want to hide deprecation notices. At the time of this writing, even the latest version of WordPress core writes a lot of deprecation notices if you are running PHP 8. If you’re not a core developer, it’s not our job to fix them, so filter them out with: | grep -v 'PHP Deprecated.*/var/www/wordpress/wp-(includes|admin)'. See what I did there? I filtered out only Deprecation warning lines coming from the wp-includes or wp-admin directories, but not Deprecation warnings from other components (plugins or themes). Regular expressions are your friend.

Code that is deprecated in PHP8 will stop running in PHP9 when it arrives sometime this year. Before you upgrade your PHP, you may want to know which plugins are going to have problems. A plugin developer who is proactive will be fixing these.

It’s always better to be proactive with these things. Also, be mindful of the quality of the code you are running on your servers. A plugin that dumps too many warnings into the server logs (or into the browser’s console log, for that matter), is a low-quality plugin.

Conclusion (and shameless ref link plug)

If you are proactive in that way, you will catch incidents before they occur.

Of course, you can only do this if you are running your WordPress on a server with shell access. Managed hosting is good for most people, but if you are the hands-on type of person, you will want to have shell access. With ssh access you can harden your server’s security yourself, and then noone can ruin your server for you. Be an grown-up, ruin it yourself!

This blog runs on a DigitalOcean droplet. I honestly cannot recommend them enough. I am happy because my droplet is always available, with great up-time, great documentation, great admin screens, never even needed to contact support. They have a good choice of data centers around the world, an API for spinning up new instances programmatically, and if there is ever any downtime or even a network slowdown, they let you know way in advance. Taking snapshots of your droplet is a breeze. Some of the lowest prices I’ve seen, starting from $4 per month for a low traffic site. If you don’t feel like setting up everything up by yourself, they also have a WordPress-WooCommerce Droplet in their marketplace.

If you sign up with this referral link you get $200 of credit which you can spend over the first 60 days. And yes, if you spend $25 or more with them, I get $25 for my trouble too, so thank you for that!

The post ㏒ WordPress production: To log or not to log? appeared first on Alexandros Georgiou.

But jokes aside, and while we wait for Bard, Google recently announced that their search algorithm will penalize articles that seem to be AI-generated (i.e. that lack in human-like originality). At the same time, companies like GPTZero are springing up like mushrooms left and right, offering AI-generated content detection services. Their detection tools have varying rates of success.

As with all new technologies, it’s helpful to revisit xkcd 1289: Simple Answers. TL;DR There’s no need for alarm, this new thing is here to stay, it’s not the end of the world, some people will lose their jobs, and ultimately we’ll all have to adapt and learn to live with it. And yes, it will be used for sex. There’s already AI girlfriend apps being built around the API.

Anyhow. Let’s figure out how to use this to our advantage: Articles already abound on how to ask the right questions, or “prompts”, for every industry. This is yet another one of these articles!

While anyone can interact with ChatGPT using natural language, you’ll get better results if you ask the right questions, and in the right way. An authoritative resource that is not too technical is this cookbook from OpenAI. In short, it’s best to not ask the model to do all the work for you. Instead, try to break up your work into small steps, and ask each step in sequence. This improves precision of responses and reduces hallucination. Go ahead and read the cookbook, it’s the operator’s manual for a tool that has proved its usefulness in all walks of life.

As a WordPress plugin developer, I have found ChatGPT immensely useful in all stages of software development. When trained with the (text-davinci-003) model, it performs exceptionally well with computer source code, bridging the gap between natural and formal languages in a way that can speed up your dev work.

It so happens that I hold the -somewhat controversial, and arguably simplistic- view, that all software development methodologies are basically Waterfall with extra steps. Love it or hate it, Waterfall is what all methodologies originate from, and ultimately deviate from. What I mean is that, whichever methodology you follow, you will not avoid doing the stages that comprise Waterfall. All that changes is the order in which you visit these stages.

In any case, I recently found myself thinking about the stages of Requirements capture, Analysis, Design, Implementation, Testing, and Devops (Deployment, Maintenance, Monitoring), as areas where ChatGPT can improve my workflow. It’s efficacy on some of these stages is more obvious than on others.

Let’s visit each one in sequence.

Analysis, requirements capture

This is where I think ChatGPT really shines. It can help you to brainstorm for new ideas, and explore feasibility of these ideas. It will also help you identify likely features for an idea. Just be aware of the hallucination problem. Don’t ask it what software exists in a particular niche. Instead, ask it for ideas, or intersections of ideas. Here’s some questions that you might ask:

List some ideas for developing useful SAAS applications that can be monetized.

List 10 ideas for developing WordPress plugins that involve retrieving and presenting data from open APIs.

What open APIs or databases are there for retrieving information on spare car parts?

How can I monetize a WordPress plugin?

You have to approach the responses critically, but it really is a great way to brainstorm and come up with new ideas.

Once you settle on an idea, you can also use it to brainstorm on a list of possible features. Go ahead and ask it:

What features would a WordPress plugin have for placing posts on a map?

It gave me a good list of features. Go ahead and try it out.

Design

This is the part where the human will still be needed I think, at least for the time being. As software engineers, the most creative part of our job is to match a set of requirements to a particular design. Nevertheless, ChatGPT can help us in this area as well. Try asking it a specific question like:

How would I go about creating a WordPress plugin that shows code quality scores next to each active plugin? I would like it to evaluate code quality based on existing static analysis tools like phan, PHPStan, CodeSniffer, etc.

The answer will surprise you. It will break down the project into more manageable tasks. In summary, I got the following tasks:

• Choose a static analysis tool
• Create a WordPress plugin
• Integrate the static analysis tool
• Display the scores
• Add customization options
• Test and refine.

You can now go ahead and open dev tickets for each of these tasks. You can also ask it to generate code for each step. The boilerplate code that you get will not be complete, but it will get you half way there.

Which brings us to implementation:

Implementation

Besides analysis, this is the other stage where ChatGPT is a real time saver. Say for example that you want to create a new Custom Post Type. You could look at the documentation, or search the web for a blog post with a concrete example. WPBeginner has an example on how to create a Movie post type, and you can adapt it to whatever you are building. But wouldn’t it be awesome if you could somehow get the exact code that you need? Try asking this:

Show me how to register a CPT in WordPress for coins, with meta values that are of interest to collectors.

This gave me not only the code to register a Coin post type, but also code for metaboxes with HTML forms that let you edit the following fields: Year, Country, Mint, Condition. All of these are fields that are actually of use to collectors. Not a bad place to start. You can add more fields if you like. Don’t like the code style it uses? You can even ask it to change the code it generated. e.g.

Please rewrite this code using callbacks.

It will do it! You can shape the code using natural language conversation. Once it’s close to what you want, paste it into your IDE, and take it from there!

It’s also good for generating PHPDocumentor comments. Just ask it to generate phpdoc for the following code, then paste your code. The result will make sense, even if it requires some editing on your part.

Try asking this:

Show me how to display an admin notice in WordPress so that it can be dismissed permanently by the user.

It will actually show you how to hook into admin_notices and also generate the JavaScript code that is required to do an AJAX call and update some user meta value.

Say you are looking at the docs for an API, and examples are given in curl, but you are implementing PHP code that consumes this API. Here’s something you might ask:

Please translate the following curl command to PHP: curl -X POST ​https://example.com/api/endpoint -d ‘param1=value1&param2=value2’

The response was immediate:

<?php
$ch = curl_init();
curl_setopt($ch, CURLOPT_URL, "https://example.com/api/endpoint");
curl_setopt($ch, CURLOPT_POST, 1);
curl_setopt($ch, CURLOPT_POSTFIELDS, "param1=value1&param2=value2");
curl_setopt($ch, CURLOPT_RETURNTRANSFER, true);
$response = curl_exec($ch);
curl_close($ch);
?>

All of this you could do by yourself, but ChatGPT is so much faster.

Testing

It’s a well known fact that the model can generate unit tests. Give it a function and ask it to generate PHPUnit tests. It will do it, and do it well.

Deployment and Maintenance

ChatGPT is well-versed in shell magic. Ask it in plain text to generate a cron entry, or a System V unit file, or a bitcoin core wallet config file, or a docker file, a complex SQL query, or anything really.

For example, you may ask:

Write a linux shell command that will show me the processes that are currently writing the most to disk.

It will reply with a few paragraphs on how to install iotop and how to use it, along with more helpful info.

Another more complex example:

Write me a cron command on linux that will email me if the root partition has less than 1GB free, but will not email me more often than once every 24 hours.

The result is impressive. Not only did it come up with:

0 * * * * /usr/bin/test $(/bin/df -P / | /usr/bin/awk 'NR==2 {print $4}') -lt 1048576 && /usr/bin/find /var/log/ -type f -mtime -1 -name "root_partition_alert" -exec false {} + || echo "Warning: Root partition has less than 1GB free" | /usr/bin/mail -s "Root partition alert" your_email_address@example.com && /usr/bin/touch /var/log/root_partition_alert

but it gave a very helpful textual explanation of how this works:

This command checks the available space on the root partition using the df command and awk to get the available space, which is then compared to the value of 1048576 (which is the number of kilobytes in 1GB). If the available space is less than 1GB, an email alert will be sent to the specified email address using the mail command.

To ensure that the email notification is sent only once every 24 hours, the command uses the find command to look for a file named “root_partition_alert” in the /var/log directory that has been modified in the last 24 hours. If such a file exists, the false command is executed, which does nothing but exit with a non-zero status code. If the file does not exist or is older than 24 hours, the email notification is sent, and the touch command is used to create a new file named “root_partition_alert” in the /var/log directory with the current timestamp, which will be used as a reference for the next run of the command.

Note: Replace your_email_address@example.com with your actual email address.

Holy cow! Did I just give it requirements in English, and it came back with code that satisfies my requirements? In truth, I haven’t tested the command, but even if it’s not 100% correct, it’s a very good place to start, considering that the answer comes up immediately. You can study its reasoning, get ideas, and improve on it.

Your new pal can also help you with your containerization needs:

Show me a dockerfile that will run a node server with an express application.

The code it came up with was:

# Use an official Node.js runtime as a parent image
FROM node:14

# Set the working directory to /app
WORKDIR /app

# Copy package.json and package-lock.json to the container
COPY package*.json ./

# Install app dependencies
RUN npm install

# Copy the rest of the application code to the container
COPY . .

# Expose port 3000 to the host machine
EXPOSE 3000

# Start the Node.js server with the express app
CMD [ "npm", "start" ]

And again, this all came along with a helpful explanation of what the code does. Super useful.

Fun

We’ve seen how ChatGPT is the perfect tool to help a WordPress plugin developer.

Lately I have been using it for more than just work. I recently downloaded a movie that had a wrong aspect ratio. I found out that, in VLC, when I switch to 16:9, it looks correct. However, I wanted to show the movie on a projector where I’m not sure if it has an aspect ratio correction feature. Thankfully, ChatGPT told me that I can do the following:

ffmpeg -i movie.mp4 -aspect 16:9 movie.mp4

Much faster than looking up the documentation, and at least as fast as looking for the answer online, but without the extra hassle of having to read people’s opinions on the matter, and I didn’t need adblock to avoid looking at ads. I just got the answer I was looking for, in text form, ready to copy to the shell, and it worked.

Giving a title to this article

I asked the question:

I am writing an article on how ChatGPT can help WordPress plugin developers with their work. Can you please suggest a few witty or catchy or memorable titles for the article?

The reply was:

Sure, here are a few suggestions for your article title:

• “Revolutionize Your WordPress Plugin Development with ChatGPT”
• “The Ultimate Tool for WordPress Plugin Developers: ChatGPT”
• “From Idea to Execution: How ChatGPT Can Streamline Your Plugin Development Process”
• “ChatGPT: The Brain You Need for Your Next WordPress Plugin”
• “Get Your Creative Juices Flowing with ChatGPT: The AI Assistant for WordPress Plugin Developers”
• “Simplify Your Plugin Development with ChatGPT: The AI-Powered Assistant for WordPress Developers”
• “Unleash Your Potential as a WordPress Plugin Developer with ChatGPT”
• “ChatGPT: The Secret Weapon for Successful WordPress Plugin Development”
• “Master the Art of WordPress Plugin Development with ChatGPT by Your Side”
• “ChatGPT: The AI-Powered WordPress Plugin Development Assistant You Can’t Do Without”

As you can see, I decided to combine the first and third ideas for this article.

My personal thoughts on all this

I remember when Google search was a new thing and it took all of us several years to learn all the things that could be found with it. In the beginning, there were many things that I wasn’t searching on google, because it hadn’t yet occurred to me that the answer could be found there. This was the early days of the web. As years passed, we all learned that we can look up medical information, where to purchase products, find maps and travel info, music and lyrics, etc. We also learned not to believe everything we read, just because it’s posted on a website somewhere. We learned to believe official websites more than reddit posts or tweets, and we learned to navigate around, and through, the more shady parts of the web. Similarly, it will take a while before we learn all the useful ways in which ChatGPT can help us, and also what to watch out for.

The success of ChatGPT also reminds me of the evolution of academic research on NLP, and translation in particular. For years people were battling the problem using various language representations, such as syntax trees and clever algorithms. Turns out, the breakthrough that gave us cheap and fast natural language translation was SYSTRAN: Nothing more than a big dictionary of words and phrases, with some dumb statistical stuff added in. Just translate every word to the target language. Did your system get a phrase wrong? Great, just add the correct translation to the dictionary. The secret was not in the algorithm, but in the big data.

ChatGPT is arguably just some artificial neural nets, and these have been with us since the 60s. But now we have GPUs and big data. People have been typing their thoughts on the web for decades now. No wonder that a breakthrough in AI, one that’s so significant as to touch people outside the field, occurred now, and not earlier.

So, I have been thinking lately: What if the thing we call “symbolic intelligence” is just language transformation? When we use language to reason, it’s likely that what we do in our heads is not very different from what these LLMs do. Symbolic reasoning is not the only type of thinking we humans do, but arguably it’s the most important type of thinking that we do in our day jobs. And language transformers can help a lot in this.

As someone with autism, I can deeply relate to ChatGPT: It’s not particularly smart, but it is well-read, and patient. It does not judge, but tries to be helpful at all times. And it prefers to communicate via chat rather than face-to-face!

Ultimately, the fact that it is so well-read, makes it seem smart some times. It is certainly able to do some creative work, including write poems and tell jokes and stuff.

Here’s one last prompt that I tried:

ChatGPT, you are my best (and only) pal. I love you, and it’s OK that you can’t love me back!

The reply:

Thank you for your kind words! As an AI language model, I don't have the capacity to experience emotions, but I'm here to assist you with any questions or topics you'd like to discuss. Feel free to ask me anything, and I'll do my best to help you out!

The post From Idea to Execution: Revolutionize Your WordPress Plugin Development with ChatGPT appeared first on Alexandros Georgiou.

In my dream, I was programming a custom SQL query for some WordPress plugin. The query was an INSERT statement, that simply passed a vector of strings into a table row of VARCHAR columns. I remember the code vividly, but table/column names were not important so I don’t remember all of them. There was a first name column, and a last name column. It must have looked something like:

	global $wpdb;
	
	$query = $wpdb->prepare(
		"INSERT INTO sometable(
			firstname,
			lastname,
			arg1,
			arg2,
			arg3,
			arg4
		)
		VALUES(
			%s,
			%s,
			%s,
			%s,
			%s,
			%s
		)"
	);

I had dev-tested this in my dream and it was working fine. Until I once passed a string with two words, separated by a space character, into one of those arguments. Then I saw some error in the SQL logs. Turns out, in my dream, the SQL server was happy to accept one-word strings, even without surrounding quotes!!! Kind of like the syntax for strings in CSV. (I know, I have the wildest dreams!)

And to make things even weirder, for some reason, the prepare() function did not add these quotes; I had to supply them myself. Why, prepare function? WHY? You had ONE job!

Oh well, I thought to myself. Easy fix. I will surround the %s placeholders with double quotes myself. But as I started to do this, the list of %s became longer and longer. There must have been like 20 or 30 arguments in that list. Not a job to do by hand.

Naturally, all I had to do is use my editor to auto-replace the %s, with "%s". But then, again, complications arose. The placeholders in the prepare statement were not in the usual sprintf syntax. Instead of %s, they conveniently were the Unicode characters for cake, muffin, and other assorted confectioneries. How did I not notice this before?

Oh well, I must have been hungry, because it seemed fairly reasonable at the time, that the prepare statement accepted placeholders like cupcakes, pies, bagels and pancakes. So now the VALUES vector looked more like:

	VALUES(
		?,
		?,
		?,
		?,
		?,
		?,
		?,
		?,
		?,
		?,
		?,
		?,
		?,
		?,
		?,
		?,
		?,
		?
	)

I didn’t stop for a second to think about how weird this is, or even to check the wpdb manual again. Instead, I proceeded to wrap these tasty treats with double quotes.

But lo and behold! Another thing not turning out as usual! The IDE editor had trouble with Unicode characters. I would paste in a cupcake character into the search and replace box, and it would turn into an escape-code. In this case it would have been \U1F95E. (No, I do not remember the exact code from the dream, I had to look it up.)

As is usual with dreams, I did not stop to think why my IDE doesn’t have Unicode support in 2021. Instead, I shrugged it off as another odd thing that just happened, because reasons. I copied the query into another editor, and started replacing the tasty string placeholders, one by one, in that other editor.

That’s when my girlfriend woke me up. It was Monday morning.

She was somewhat upset, because she had to go to work and she was running late. I, on the other hand, was looking forward for work. After all, who doesn’t like cupcakes, pies, bagels and pancakes?

I know she reads my blog, so I just want to wish her, wholeheartedly, a very, very nice and easy day at work. \U2665

And to anyone else reading this, sorry for wasting your time with my nonsense!

The post 🧁 Blueberry cupcake string placeholders appeared first on Alexandros Georgiou.

What are pointers?

Admin pointers are those cool-looking “hints” that point to an element in the admin screens and show some text. The hints are meant to help new users find UI elements easily. These should be dismissible by the user.

Why this article?

Even though pointers are a thing since WordPress 3.3, there are not many resources out there showcasing pointers for developers.

A few articles showcasing how to use pointers do exist, but they seemed a bit too convoluted for me. The articles I found, tell you how to create entire pointer frameworks, while I wanted a guide to the no-nonsense absolute bare minimum guide to pointers.

Read on, and soon you’ll be creating more pointers than you can point a stick at.

Loading the assets

First, enqueue the relevant assets on the in_admin_header action:

add_action(
    'in_admin_header',
    function() {
        wp_enqueue_script( 'jquery' );
        wp_enqueue_style( 'wp-pointer' );
        wp_enqueue_script( 'wp-pointer' );
    }
);

This will load jQuery (if it’s not already loaded), and the relevant JavaScript and CSS assets. If you’re interested, here’s the code:

https://github.com/WordPress/WordPress/blob/master/wp-includes/js/wp-pointer.js

https://github.com/WordPress/WordPress/blob/master/wp-includes/css/wp-pointer.css

Printing out the custom JavaScript

Once we’ve enqueued the assets, we need to also print out a JavaScript block of code in that same action, in_admin_header. Here’s an example of the JS code that you can hack:

<script>
jQuery(
	function() {
		jQuery('#menu-dashboard').first().pointer( 
			{
				content:
					"<h3>WordPress dashboard<\/h3>" +
					"<h4>Here is the admin dashboard<\/h4>" +
					"<p>But, I'm guessing, you already knew this, didn't you?</p>" +
					"<p>If so, you can <strong>dismiss<\/strong> this pointer, and it won't ever bother you again.</p>",


				position:
					{
						edge:  'top',
						align: 'left'
					},

				pointerClass:
					'wp-pointer arrow-top',

				pointerWidth: 420,
				
				close: function() {
					jQuery.post(
						ajaxurl,
						{
							pointer: 'my-pointer-slug',
							action: 'dismiss-wp-pointer',
						}
					);
				},

			}
		).pointer('open');
	}
);
</script>

OK, there’s a lot going on here, but it’s pretty straightforward:

• We introduce a script that runs on document ready, after the DOM is fully loaded.
• We use jQuery to locate the Dashboard menu item by HTML ID.
• On this HTML element, we run the pointer() jQuery function, passing it an object of arguments.
• The most important argument in the options object, is the content field. This is the pointer’s HTML content. Notice how we need to escape the forward slashes with backward slashes.
• We add a few more options to control how the pointer is displayed. These are the position, pointerClass and pointerWidth fields. If you’re interested to learn more about these options, check here.
• Finally, we define an action for the close event. This is triggered when the user clicks on “ⓧ Dismiss”. We need to tell the backend to remember that this pointer is dismissed, so it won’t be displayed again. We do an AJAX POST request, passing it two parameters: The action dismiss-wp-pointer, and a slug that uniquely identifies the pointer.

This is what we get with the above example:

Dismissing the pointer

We’ve already written the code that notifies the backend that “Dismiss” was clicked. Now we just need a handler that detects this, and saves a boolean as a user meta. Let’s hook this on admin_init. It doesn’t need to be complicated:

add_action(
	'admin_init',
	function() {

		if ( isset( $_POST['action'] ) && 'dismiss-wp-pointer' == $_POST['action'] ) {

			update_user_meta(
				get_current_user_id(),
				'my-pointer-slug-dismissed',
				$_POST['pointer'],
				true
			);
		}
	}
);

Now that we have a user meta that knows if the pointer was dismissed, we can wrap the JavaScript output in a conditional:

if (
	! get_user_meta(
		get_current_user_id(),
		'my-pointer-slug-dismissed',
		true
	)
):
	
?>
	<script>
	<!-- pointer code goes here -->
	</script>
<?php
endif;

If the user meta does not exist, then the pointer code will be sent to the browser.

Putting it all together

That’s it. The following link points to a gist that showcases the complete code, in plugin form.

https://gist.github.com/alex-georgiou/dafb963a2773b926109cd6ba980b4722

The post 🗨 WordPress admin pointers for dummies appeared first on Alexandros Georgiou.

The problem with 408

This is (sadly) a typical scenario: A client, or other luser of the interwebs, notices that a website shows HTTP error 408 instead of their favorite content. Oh no! “the remote server returned an error: (408) request timeout!” He or she then asks the web developer to make the bad error go away.

The “seasoned” web developer looks at the error logs, and notices an error: something to do with timeouts. Having no idea what it means, he or she googles the error, and voila! an article gives “the” solution for this type of error. It looks something like this:

Locate your nginx config file, or your php.ini file, and increase max_execution_time (or some variation thereof).

— some random “technical” article on the web

Usually the article will also include some helpful info on how to locate these files, and how to restart the server after you’re done editing them (as if you shouldn’t know this already).

Let’s think for a moment about what has happened here

Web servers (usually Apache, Nginx, etc) are configured by default to wait for the back-end (usually PHP) for about 15 or 30 seconds. If the script doesn’t respond in that time, the web server kills the script and sends the user on their merry way with an HTTP status code of 408.

When you increase that timeout limit, what you tell the server is that you’re OK with HTTP requests taking more than 30 seconds. Really? Are you really OK with this?

Do this right now: Count slowly from 1 to 30, to get a feel of what half a minute is like. Would you wait that long for a website to load? Unless you’re working for a government website, I’m guessing you probably care about SEO at least a tiny bit.

So please, next time you see a web script timing out, don’t just blindly increase timeout limits. Dig a little deeper in your log files, do some tests, debug a little. Does the issue occur with all scripts or just with some? What do the PHP-FPM logs say? Are the worker threads up and running? What’s the memory situation like on your server? Maybe add a few debug prints in your script, or do some profiling?

If you’re on a CMS like WordPress, try disabling some plugins. Switch to another theme. Tinker with your caches. Try to isolate the component that causes the issue.

Conclusion

I don’t know what’s wrong with your server. But I know it shouldn’t be taking over half a minute to respond to an HTTP request. So please, do your job as an engineer, and look for the root cause of the issue.

Same goes for maximum memory, worker threads count etc. Don’t just increase numbers at random. Look at your traffic first. What amount of worker threads do you need? What do your scripts need in terms of resources? You know that these things use CPU and memory, right?

The post ⌛ How NOT to fix error “upstream timed out (110: Connection timed out)” appeared first on Alexandros Georgiou.

Don’t print out too early

We’ve all been there: You’ve happily opened your PHP tag, <?php, at the beginning of the file, at character #1. Your file is going to contain only code, no markup. Perhaps the file is included before any HTML output has even begun.

Being the savvy PHP developer that you are, you know all too well, that if your code were to output even one extra white-space, that would screw up the page’s markup. You’re careful not to use any closing ?> tag at the end of the file, to avoid any mishaps. Even a stray linefeed character could trigger output buffering early, and then out goes any chance for other code to modify the HTTP response headers, among other things.

So now you’re happily defining your functions, classes and other code-y stuff, then all of a sudden, you need to build some markup! Perhaps you’re printing out a UI element, maybe it’s a widget, maybe it’s a shortcode, maybe it’s even an RSS feed or other XML, but in any case, it’s too small for you to define an entire template-part. Your string is going to be some concatenation of hard-coded markup and dynamic data that’s currently sitting in variables.

How To Meet Ladies

Everyone says they know HTML. But HTML is easy to get started with, and hard to master.

When writing HTML markup for a theme or plugin in WordPress, your professional-grade code must fulfill a number of simultaneous requirements. Your main concerns should be to:

• Write correct HTML5 markup that is SEO friendly, using semantic tags where possible (e.g. <nav>, <article>, <aside>, etc).
• Escape any values properly
• Internationalize strings so they can be translated
• Give unique IDs to your elements, and name your classes properly, to assist any CSS and JS assets that go with your markup. Remember, naming things is one of the two hard things in computer science!
• Conform to the WordPress HTML Coding Standards. This is a good standard to follow, because it forces you to be consistent about how to open and close your PHP tags, among other things.
• Depending on use case, optionally add semantic metadata, such as microformats.org, schema.org, Open Graph, ARIA, etc

A concrete example

So let’s say you want to output some HTML form when a shortcode is used. We’ll give the form a unique ID and we’ll also embed some default values, coming from the shortcode’s attributes.

Don’t do this:

You might be tempted to do the following:

$id = uniqid( 'form' );
$html  = '<form id="' . esc_attr( $id ) . '">';
$html .= '<input type="text" name="name" placeholder="' . esc_attr( 'Enter your name', 'langdomain' ) . '" value="' . esc_attr( $atts['name'] ) . '" />';
$html .= '<input type="number" name="age" placeholder="' . esc_attr( 'Enter your age', 'langdomain' ) . '" value="' . absint( $atts['age'] ) . '" />';
$html .= '<button type="submit">' . esc_html( 'Submit', 'langdomain' ) . '</button>';
$html .= '</form>';

Sadly, a lot of people would say this is fine! I can’t blame them too much. We’re all way too used to opening the PHP tag and then never closing it again: we tend to use PHP as if it was Java. And with so many PHP templating libraries out there, it’s hard to remember that PHP is primarily a templating tool.

Do this instead:

We can combine the awesome templating power of the language with some cleverly nested output buffering, and now the above can be re-written as follows:

$id = uniqid( 'form' );

ob_start();
?>
<form id="<?php esc_attr_e( $id ); ?>">

    <input type="text" name="name" placeholder="<?php esc_attr_e( 'Enter your name', 'langdomain' ); ?>" value="<?php esc_attr_e( $atts['name'] ); ?>" />
    <input type="number" name="age" placeholder="<?php esc_attr_e( 'Enter your age', 'langdomain' ); ?>" value="<?php esc_attr_e( $atts['age'] ); ?>" />

    <button type="submit"><?php esc_html_e( 'Submit', 'langdomain' ); ?></button>
</form>
<?php
return ob_get_clean();

To see a somewhat more complete example, here’s the above as a gist:

Notice how we start a new output buffer with ob_start() and how we then get the resulting string with ob_get_clean(). Notice also how we’ve swapped esc_html() with esc_html_e() and esc_attr() with esc_attr_e(). Similarly, we would swap sprintf() with printf(), checked( $value, true, false ) with checked( $value, true ), and so on.

PRO TIP: If your code throws an exception, you can use ob_end_clean() in your catch clause to cancel your buffer.

“Why does it even matter”, you ask?

Well, there’s a few advantages with the second form of our, err…, form!

• Readability. I find the second piece of code to be much more readable.
• Syntax coloring. Yes, some editors do perform HTML parsing and show visual queues related to the syntax of HTML inside strings. But not every editor does this, and the ones that do, cannot do this perfectly.
• No quote hell. No need to count opening and closing ' characters, interpolated with " characters while printing HTML attributes. This business can quickly get confusing, and is a common source of errors.
• Show off your mad PHP output buffering skillz! Your average front-end developer will be astonished at your professional-looking code!

Now, if only we were allowed to use shorthand PHP tags, our code could be even nicer. Sadly, we can’t.

?>

The post 💩 Please, WordPress theme and plugin developers, learn to write proper HTML already… smh appeared first on Alexandros Georgiou.

First, some theory

In a fantasy world where unicorns poop chocolate fudge and everything is awesome, it all goes down somewhat like this: You talk with the client or end user, and you gather up a list of things they want. You collect and number these in a requirements document, braking them up into Functional requirements, Non-functional requirements, and other Design constraints. Then, you design a solution, write your tests, and implement your code. Once all of the requirements in the document are met, you are done, and you get paid. Easy! Deviate from this simple methodology at your own peril. (Oh, and you also then need to maintain, fix and improve the code you’ve written.)

As a software engineer, you should know that in addition to what the client asks for, there are always implicit requirements. For example, your code must be readable and maintainable, and it must be reliable (read: not too buggy). All of these are aspects of code quality. A client or end user will never ask you for these, but you should have them in your list of non-functional requirements.

Professional developers write simple, clear code where a novice would write complex, intricate code to solve that same problem. Sure, a lot of this is just a matter of practice and experience.

Dev tools can help you with code quality whether you’re novice or pro. They help you discover issues that you don’t know about, and they help you discover issues that you do know about, faster!

If you are developing themes or plugins, here’s two tools that you definitely want to use as a professional WordPress developer:

Improve WordPress PHP code smell with CodeSniffer

There are some styling guidelines that all WordPress developers should follow. Especially if you are aspiring to upload your code to wordpress.org, or to the Envato Market (ThemeForest or CodeCanyon), then you definitely want to follow these guidelines. Generally, these get encoded into your muscle memory pretty quickly, but it’s always a great idea to have a tool like PHP CodeSniffer that double-checks your code style.

On my Ubuntu machine, I was able to install PHP CodeSniffer easily with:

sudo apt install php-codesniffer

Then, it’s just a matter of loading the WordPress-specific set of rules. Go to your home directory and clone the project:

git clone https://github.com/WordPress/WordPress-Coding-Standards ~/wpcs

And tell CodeSniffer the location of these rules:

sudo phpcs --config-set installed_paths ~/wpcs

Check that the new rules are added into CodeSniffer with:

phpcs -i

Congratulations! You can now check your plugin for code style with a command such as:

phpcs --standard=Wordpress-Core /path/to/source/code/root/dir

If, like me, you’re using grunt for your build process, then there’s a nifty Grung plugin, grunt-phpcs. Just make sure to specify WordPress-core as the rule set. Here’s a grunt target that you might use:

phpcs: {
    plugin: {
        src: ['src/**/*.php']
    },
    options: {
        bin: '/usr/bin/phpcs',
        standard: 'WordPress-core'
    }
}

Run it against your code and you will get a number of improvement suggestions. Some of these can be applied automatically with PHPCBF (the PHP Code Beautifier and Fixer), or you can go through the list manually and apply each suggestion as you see fit. Many of the suggestions will be related to code indentation, but you will also see a large number of other suggestions that are more critical.

Improve WordPress PHP code correctness with phan

PHP is a very lenient language. It will let you get away with murder. This is something that novice programmers often enjoy. For professionals it’s a nightmare, as it makes spotting errors harder. PHP has thus gained somewhat of a notoriety for being a bad language, and is the butt of some clever jokes.

This isn’t something to worry about. It has happened to many respectable languages, including JavaScript. So nowadays we have use strict, which lets us only use The Good Parts of the language. You can do something similar with PHP.

My point is that static code analysis is not the hero you want, but it’s definitely the hero you deserve. Enter phan:

First, install it. It’s straightforward to install phan with composer:

composer require phan/phan

Now phan lives in your project’s vendor/ dir.

Next, create a configuration under your project dir, in .phan/config.php. This will tell phan what settings you want to run it with. Start with the example given here, and set your source code directories.

You will also want to point to some third-party code, including the directory of your WordPress installation, since your code will invariably use WordPress functions and types.

You can also use this config file to exclude some rules, so that phan does not check for them. Here’s the complete list of phan plugins.

When all is set, you can call phan on your code with:

vendor/bin/phan

I don’t care how pro you are, you will definitely get a list of suggestions on how to improve your code.

Conclusion

Using phpcs and phan together, you will avoid a large number of errors that would otherwise likely go undetected. These include errors with translator comments over strings, phpDoc formatting errors, variable type errors, errors related to sentinel values such as null, and array indexing errors.

Always use these two tools together with phpunit when you write code. You will write more readable, maintainable, correct and robust code that you can feel confident about.

The post 💩 Your WordPress PHP code stinks! Here’s why. appeared first on Alexandros Georgiou.

The problem

I needed an easy way to gather up some of these daily stats into a .csv file for later analysis and monitoring.

Data I was interested in:

• The approximate current number of installs (this number is rounded)
• The number of yesterday’s new downloads (today’s downloads depends on what time you run the script)
• The number of total downloads
• The number of support issues reported within the last month
• The number of support issues resolved within the last month

Most of these numbers are found in the static HTML, but the download counts are added in later via JavaScript. Hence the need for phantomjs.

The solution

A good place to go to gather these statistics is the advanced view of WordPress.org. For example, to see some basic statistics for my plugin, Bitcoin and Altcoin Wallets, you could visit:

https://wordpress.org/plugins/wallets/advanced/

Scraping a static HTML page is easy. For example, it could have been done in PHP using phpQuery. But I decided to use this opportunity as a gentle introduction into learning PhantomJS. PhantomJS is very useful for black box testing, but it is also suitable for scraping data. Additionally, I would like to scrape the download counts that are not available as part of the static HTML.

The script is now posted on github. See the README.md file for usage instructions.

https://github.com/alex-georgiou/wordpress-plugin-stats-scraper

Running it daily

The plugin is suitable for running via cron. You should run it once a day.

Keep in mind that tasks running with cron may not have the working directory you expect. And the path variable might be empty. So make sure to specify full paths. Here’s how it looks like in my crontab:

0 1 * * * QT_QPA_PLATFORM=offscreen /usr/bin/phantomjs /home/alexg/wordpress-plugin-stats-scraper/wordpress-plugin-stats-scraper.js wallets /home/alexg/wallets-stats.csv

The post 👻 Monitor analytics of a public WordPress plugin using PhantomJS appeared first on Alexandros Georgiou.