HOW TO: Find the IP address of a domain from the local server before propagation

Have you ever wanted to know the ip address assigned to a domain according to your local server? I had this problem. I needed to get an IP address of a domain before the domain propagated. I was writing a custom cpanel postwwwacct script. For some very, very odd reason, cpanel will tell you IF you have a dedicated ip address for the domain, but not what it is. Fortunately postwwwacct occurs after the bind of the ip address.

I thought I would have to use a grep and awk to parse the IP address out of the .db file in /var/named but fortunately it occurred to me that I could just do a simple dig call within my script and get the value back. That’s straight forward and returns only the information I want to have.

So here is a perl script that does this:

#!/usr/bin/perl
$ns='ns1.example.com';
$domain='wahoo.com';
$ip=`dig \@$ns $domain A +short`;
if ( $ip == -1 )
{
  print "command failed: $!\n";
}
else
{
  printf "The result is: " . $ip;
}

 

You will of course have to replace ns1.example.com with the url for the name server running on your server.
The backticks (“) indicate you need to run a command line and return the stdout results. (And yes, I think backticks sound gross but that’s what they are called.)

Also note the @ is a special character and must be escaped with a \.

Hope this helps!

HOW TO: Dump or Backup all MySQL databases to separate files

I needed to transfer about 40 databases from a new clients server over to my hosting platform.

When doing a couple database exports, I might use PHPMyAdmin to do the export. Heck, it’s convenient because I usually want to look around at the same time. For doing a straight export, mysqldump is a great program that is even easier than phpmyadmin. There’s really nothing to it! (NOTE: For large databases with a LARGE number of tables you need to add –skip-lock-tables)

mysqldump –all-databases > dbdump.sql

If you want to get fancy, you can even compress the file

mysqldump –all-databases | gzip -9 > dbdump.sql.gz

But I wanted to go one step further. I wanted every database to have a separate file, compressed and correctly named.

I was shocked how simple it was to write. First I had to ask mysql for a list of the databases. Then I needed extract just the database name, removing the table borders. Then I needed to tell mysqldump to use each database name in the export source and destination.

In no time I had a 3 line script that was incredibly powerful.

I give you backupdbs.sh

#!/bin/bash
for database in $(mysql -e "show databases"|awk -F " " '{print $1}'); do
  mysqldump $database | gzip -9 > $database.sql.gz
done

 

 

Note that you may need to provide the -u and -p parameters (username and password).

That’s all there is to it!

Answered: Help! WordPress keeps missing my scheduled posts!

I gave a new client a totally wrong answer the other day.  Well, it was right as far as I always knew.  Scheduled posts have been a bane to a bunch of blogs I’ve seen. I’d thought that there were bugs in wp-cron, that had been fought for years and years by dozens of people and no one had yet found the right fix.. I just stumbled across this post to the support forum made a year ago by Otto, well known WordPress Guru Supreme, that proved I was completely mistaken. With the additional source of completely swamped servers, I think this post explains every source of recent missed posts I’ve seen:

Short answer: Add this to the to defines in your wp-config.php file:

define(‘ALTERNATE_WP_CRON’, true);

Really long answer, for masochists: Scheduled posts are not now, and have never been, “broken”. The developers of WordPress cannot fix it because there is nothing to fix. The problem lies in the fact that your server, for some reason, cannot properly execute the wp-cron process. This process is WordPress’ timing mechanism, it handles everything from scheduled posts to sending pingbacks to XMLRPC pings, etc. The way it works is pretty simple. Whenever a WordPress page loads, internally WordPress checks to see if it needs to fire off wp-cron (by comparing the current time with the last time wp-cron ran). If it does need to run wp-cron, then it tries to make an HTTP connection back to itself, calling the wp-cron.php file. This connection back to itself is there for a reason. wp-cron has a lot of work to do, and that work takes time. Delaying the user seeing his webpage while it does a bunch of stuff is a bad idea, so by making that connection back to itself, it can run the wp-cron program in a separate process. Since WordPress itself doesn’t care about the result of the wp-cron, it only waits a second, then goes back to rendering the webpage for the user. Meanwhile, wp-cron, having been launched, does its work until it’s finished or it runs out of execution time. That HTTP connection is where some badly configured systems fail. Basically, WordPress is acting like a web browser. If your site was http://example.com/blog, then WP will call http://example.com/blog/wp-cron.php to start the process. However, some servers simply can’t do that for some reason. Among the possible reasons:

  • Server doesn’t have DNS, and so it can’t figure out who “example.com” is, even though it is *itself*.
  • Server administrators, in a misguided attempt at security, have blocked “loopback” requests, so it can’t actually make a call back to itself.
  • Server is running something called “mod_security” or similar, which actively blocks the call due to brain-dead configuration.
  • Something else.

The point is that for whatever reason, your web server is configured in some non-standard way that is preventing WordPress from doing its job. WordPress simply can’t fix that. However, if you have this condition, there is a workaround. Add this to the to defines in your wp-config.php file:

define(‘ALTERNATE_WP_CRON’, true);

This alternate method uses a redirection approach, which makes the users browser get a redirect when the cron needs to run, so that they come back to the site immediately while cron continues to run in the connection they just dropped. This method is a bit iffy sometimes, which is why it’s not the default.

I would add three things to this explanation:

  • mod_security can be configured to block the ip address of the server, the domain name itself or even wp-cron specifically.  You can do some testing of this by doing a wget for your domain from your server. (wget http://example.com/wp-cron.php) and look at the results.  If there’s a problem and you can’t change the mod_security config itself, the fix may be configuring .htaccess to specifically allow from your server’s external IP address access to that file.
  • The loopback address failure can be caused by the Linux networking subsystem failing to load completely. Diagnosing that is off topic here, but you could look for it and see if you can “ping 127.0.0.1” and your domain from your server. If that works, loopback config isn’t the problem. If you it doesn’t, try running “ifconfig lo up” and doing it again. If it works, you’ve got some networking issues to resolve. If once device fails due to a config error, the rest don’t come up in systems I’ve seen.
  • It has been reported and disputed that the line in cron.php that calls the script has  a timeout that is too short or a value that is incompatible with some configurations. I also suspect that this can be a problem on extremely overloaded systems.  However, the fix might be worse than the original symptom.

The line in question is:

wp_remote_post($cron_url, array(‘timeout’ => 0.01, ‘blocking’ => false));

Some people have changed 0.01 to 1 and it works however, this adds a full second to every page load that calls wp-cron – and that could be very costly Google wise.  So be conservative  using this “fix”. On your overloaded system, adding milliseconds or seconds or more might not be noticeable, but then again – you probably have more pressing things you should be worrying about than a few missed posts.

HOW TO: Add menus to the WordPress 3.1 admin bar

This was posted to the wp-hackers list by Frank Bültge and I didn’t want to lose it.  Here is how to add a new menu item to the admin bar:

function wp_codex_search_form() {
  global $wp_admin_bar, $wpdb;

  if ( !is_super_admin() || !is_admin_bar_showing() )
    return;
   $codex_search = '<form target="_blank" method="get" action="
http://wordpress.org/search/do-search.php" style="margin:2px 0 0;">
 <input type="text" onblur="this.value=(this.value==\'\') ? \'Search the
Codex\' : this.value;" onfocus="this.value=(this.value==\'Search the
Codex\') ? \'\' : this.value;" maxlength="100" value="Search the Codex"
name="search">
 <button type="submit">
<span>Go</span>
 </button>
</form>';

 /* Add the main siteadmin menu item */
  $wp_admin_bar->add_menu( array( 'id' => 'codex_search', 'title' => 'Search
Codex', 'href' => FALSE ) );
   $wp_admin_bar->add_menu( array( 'parent' => 'codex_search', 'title' =>
$codex_search, 'href' => FALSE ) );
}

add_action( 'admin_bar_menu', 'wp_codex_search_form', 1000 );

The answer to: How do you set directories to 755 and files to 644?

AKA: Choosing and setting safe file permissions for a WordPress install.

 

If you have ever watched a WordPress security presentation, you’ve heard the advice:

Generally speaking, directories should be 755 and files should be 644.

But the presenter never tells you how to change these settings for all the directories and files in the WordPress install. I’m guilty of the same thing in my presentation.  After all, Bash doesn’t go over well in front of general audiences.

But how do you change all the directories and files to have the settings that you want? You’re certainly not going to go around and manually change all of the permissions file by file & directory by directory!  And if you did, would you still be able to upload files and upgrade your plugins with those permissions? Probably not.

I thought I would share how I handle this on my servers.  My method is not the most refined, and does have some minor issues, but it works and gets me 95% to where I should be.

I have this function built into my .bashrc file:

function defaultr()
{
find . -type d | xargs chmod -v 755
find . -type f | xargs chmod -v 644
find . -type d -iname uploads | xargs chmod -Rv 774
find . -type d -iname blogs.dir | xargs chmod -Rv 774
find . -type d -iname cache | xargs chmod -Rv 774
find . -type d -iname themes | xargs chmod -Rv 774
find . -type d -iname plugins | xargs chmod -Rv 774
find . -type d -iname upgrade | xargs chmod -Rv 774
}

It is meant to be run from a www or public_html directory with WordPress installed under it.  This will follow the basic rule, and modify the special directories so that you can upload, upgrade and use a caching tool.  Additionally, it works for WordPress with multi-site either enabled or disabled.
Now, if you wanted to have some real fun, you could add lock and unlock functions so that there can be no changes to your plugins and themes unless you want them to be changed and then your site IS secure.

function lock()
{
find . -type f -iname wp-config.php | xargs chmod -Rv 644
find . -type d -iname themes | xargs chmod -Rv 754
find . -type d -iname plugins | xargs chmod -Rv 754
find . -type d -iname upgrade | xargs chmod -Rv 754
}

function unlock()
{
find . -type f -iname wp-config.php | xargs chmod -Rv 774
find . -type d -iname themes | xargs chmod -Rv 774
find . -type d -iname plugins | xargs chmod -Rv 774
find . -type d -iname upgrade | xargs chmod -Rv 774
}

Of course that means you need to be confident and not panic when you see messages like “Cannot update/modify/create x” and remember to unlock things before making those changes, but largely, you can set and forget these things until you are ready to apply updates.

One final note. The numbers in these examples are suitable for many shared hosts. If you pay more than $10/month for hosting, chances are you will be able to changes those numbers to better suit your needs. When I lock the files down, I use 544 and sometimes lower.

In any case, this is a good baseline for your functions and will get the job done.  If you have a similar/better technique, I’d love to hear it.

(BTW have you figured out where the flaws are?)

Addendums:

  • I should mention that the settings here are verbose.  You can change the -Rv to -Rcf or just -Rf and get the same results less space taken up.
  • If you get an error like “chmod: missing operand after ‘744’” This is “normal”. It just means that your install does not have one of the directories. Maybe you’ve never upgraded a plugin and the upgrade directory doesn’t exist. Or you have never run multisite and blogs.dir does’t exist.  You can either remove these lines or create empty directories, if the errors bother you.
  • I’ve added echo lines so I can determine which commands create errors

In Linux, how do you unzip all the zip files in a directory?

This is a simple issue that trips me up every now and then.  Let’s say you have a directory of a couple dozen themes all in zip files.  It would be a pain to type “unzip filename.zip” for every single one, but when you do an “unzip *.zip” you get something like:

[root@hosting ~]# unzip *.zip
Archive: file1.zip
caution: filename not matched: file2.zip
caution: filename not matched: file3.zip
caution: filename not matched: file4.zip

You see the problem is that as command line runs, the * is processed redundantly via both unzip and the *nix CLI. In the end, it tries to extract from the first file it finds, all of the remaining files in the directory by name.

The trick to fix this is simple. Escape the * with a back slash when making the call. Then the * is passed as a character to unzip and nothing more.  Like so:

[root@hosting ~]# unzip \*.zip
Archive: file1.zip

inflating: …

Escaping symbols and aliases is important in other circumstances as well.  Every now and then a Linux command simply won’t work because the CLI has replaced text mid command with other text.  I once couldn’t FTP in somewhere because the password was replaced before FTP could actually run. Escaping the symbols in the password involved would have allowed the connection to work..

Adding a line to /etc/hosts via bash

It is easy to append a line to a file in BASH.  I wrote a short script this morning that I can use to add a single line to my servers /etc/hosts pointing any domain I want at the loopback IP address of 127.0.01.

So I thought I would share..

#!/bin/bash
SUCCESS=0                      # All good programmers use Constants
domain=example.com             # Change this to meet your needs
needle=www.$domain             # Fortunately padding & comments are ignored
hostline="127.0.0.1 $domain www.$domain"
filename=/etc/hosts

# Determine if the line already exists in /etc/hosts
echo "Calling Grep"
grep -q "$needle" "$filename"  # -q is for quiet. Shhh...

# Grep's return error code can then be checked. No error=success
if [ $? -eq $SUCCESS ]
then
  echo "$needle found in $filename"
else
  echo "$needle not found in $filename"
  # If the line wasn't found, add it using an echo append >>
  echo "$hostline" >> "$filename"
  echo "$hostline added to $filename"
fi

What is CGI and What is FastCGI?

There was a long rant in the wp-testers mailing list today talking about how we weren’t helpful to someone who was getting a “500 Internal Server Error” message after installing WordPress.  After complaining about our helpfulness and some bragging about past income levels, the poster said:

Quickly about the Internal Server Error.  It most probably has to do with using fast graphics on a server.  Older version of Apache like 1.333 are still in use with major hosting sites.  Using fast graphics with this slopped together code from wordpress will quite often cause  500 errors by causing the server to time out on you.

I think the writer misunderstood the meaning of FastCGI.  I wrote this reply/post to help clarify it for him and for myself as well:

We old school programmers remember the term CGI as “Computer Graphics Interface” and how it related to the initial concepts that matured into the video card drivers supplied with Windows. In DOS, the games you ran each came with their own hand written drivers for the most popular cards. CGI was a standard for writing these drivers. I still have a CGI programming book on my shelf that talks about programming for the Trident, Matrox and Tseng Labs video cards. As a side note, CGI in the movie industry refers to Computer Generated Images – also graphics related.

But when talking about Web Servers, CGI is the Common Gateway Interface. It really has little or nothing to do with graphics. The CGI is basically how the server turns website requests into the execution of console programs and the passing of the results back to the browser. In my experience, straight CGI programming is most often done in PERL, but doesn’t have to be. CGI programs are separate and independent from the Apache server. They are executed and return a result. That’s it.

FastCGI is an extended version of the CGI that is optimized for speed and scalability. It has also been extended to allow the web server to run on one machine and the CGI calls to execute on another.

Where all this comes into play with PHP (and WordPress) is that PHP can be connected with Apache (or lighttpd or w/e) in different ways. It can be a module interfacing through Apache API calls. Or it could also be a CGI application. If connected to Apache via CGI, the PHP code is executed by the server as console calls to a separate PHP application. Since FastCGI is compatible with CGI, FastCGI can also be a chosen interface between PHP and Apache or whatever webserver you are using.

As all of this changes how PHP and your webserver talk, over the years there have been bugs in WP that were specific to FastCGI servers. However, a 500 Internal Server Error is simply the server saying “OK This is all screwed up and I have no idea what to do now.”. Yeah, maybe that has something to do with how FastCGI is setup, but it is even more likely that you are simply missing a bracket or comma in your .htaccess file.

So even if this question had been in the right forum, we couldn’t have helped the person.  When the question is ‘My server says “Something is screwed up”. How do I fix it?’, the best response we can offer will start with  “Ummmmm”.

Also, I’ve come to understand this stuff just through passing conversations with SysAdmins and random reading. So please correct me if I am wrong on any of this.

Alternate version of “The Loop”

OK So, most theme developers are aware of what the loop is… it displays your WordPress posts.

The one interesting thing about the loop is that it bounces in and out of the php and straight html output.  So, most of the functions that are called do the echoes themselves.  But what if need the output in a variable, or want to code it so that the call is part of its own echo function? You simply cannot use the same functions.

Fortunately WordPress provides two versions  of each of these core functions.  For example there is the_content and get_the_content() also the_permalink() and get_permalink() (yes, that inconsistency has ALWAYS bothered me..)

Here is a version of  “The Loop” which calls all of the alternate versions of the functions:

if (have_posts()) {

while (have_posts()) {

the_post();
echo ‘<div id=”post-‘ . the_ID() . ‘”>’;
echo ‘ <div></div>’ . “\n”;
echo ‘ <div>’ . “\n”;
echo ‘ <div>’ . “\n”;
echo ‘ <h2><a href=”‘ . get_permalink() . ‘” rel=”bookmark” title=”Permanent Link to ‘ . the_title_attribute(‘echo=0’) . ‘”>’ . the_title(‘echo=0’) . ‘</a></h2>’ . “\n”;
echo ‘ <small> <!– by ‘ . get_the_author() . ‘ –></small>’ . “\n”;
echo ‘ <div>’ . “\n”;
$content = get_the_content();
$content = apply_filters(‘the_content’, $content);
$content = str_replace(‘]]>’, ‘]]&gt;’, $content);
echo $content . “\n”;
echo ‘ </div>’ . “\n”;
echo ‘ </div>’ . “\n”;
echo ‘ </div>’ . “\n”;
echo ‘ <div></div>’ . “\n”;
echo ‘</div>’ . “\n”;

}

echo ‘<div>’ . “\n”;
echo ‘ <div>’ . next_posts_link(‘&laquo; Older Entries’) . ‘</div>’ . “\n”;
echo ‘ <div>’ . previous_posts_link(‘Newer Entries &raquo;’) . ‘</div>’ . “\n”;
echo ‘</div>’ . “\n”;

} else {

echo ‘<h2>Not Found</h2>’ . “\n”;
echo ‘<p>’ . “Sorry, but you are looking for something that isn’t here.” . ‘</p>’ . “\n”;
get_search_form();

}

Determining what versions of WordPress you are hosting

If you host lots of different sites for people, one of the things you might want to know is what versions of WordPress each site is running.

WordPress stores the version number in a variable named $wp_version which is set in the file version.php.

With that information in hand, you can write a bash command that you run from your /home directory to display all of the WordPress versions you have on your server:

find . -name version.php -type f|xargs grep ^\$wp_version

This is one of the aliases I have in my .bashrc file.