This latest version of ShieldPRO brings major improvements to Shield’s scanning architecture with 3 primary aims:

  • simplification
  • performance
  • background processing

In this release article we’ll outline everything that’s changed to make the Shield scans better than ever before.

#1 Shield Security Scanning Simplification

The existing structure of the scanning system resulted from the evolution that the plugin has taken over the past few years.

Over time we added more areas to scan and each time we did so, we created a new “scanner” to handle it. Before ShieldPRO 13 we had 4 separate file scanners.

We’re looking to add further scanning areas and this would have led to even more scans being added. So we had to take a step back and do a full review our approach.

Another issue with the file scanners was that they were being run completely independently.

As an example, the plugin guard would examine plugin files, then separately the malware scanner would run over the same files again looking for malware. We were wasting resources and execution time. The separate approach also made it more difficult to link results for the same file, from different scans, together.

So instead, we’ve created a single filesystem scanner which is a combination of the 4 scanners:

  • WordPress Core File Scan
  • Unrecognised Core File Scan
  • Plugin/Theme Guard Scan
  • Malware Scan

It’s now simply called the “WordPress File Scan” and when this runs, it’ll run all the available scanners together in 1 single pass, including any extras we add further down the line.

The scanner also tightly integrates with our ShieldNET Hashes API, meaning results are far more accurate, especially for premium plugins and themes.

#2 Shield Scanning Improvements: Performance

As you can imagine, combining the 4 file scanners into 1 scanner results in significant performance improvements already.

Each file on-disk is processed just the once.

Alongside the actual processing of the scan items, there’s the data storage and management aspect of scans. We’ve completely scrapped the older data model and built brand new database tables to hold all the data required. The new database tables are far more flexible and by scrapping the 4 separate file scanners, we can more efficiently store results data without duplication.

We’ve also adjusted our SQL queries so they’re more granular and fewer queries are required overall.

#3 Shield Scanning Improvements: Background Processing with WP-CLI

This is probably one of our best enhancements to the scanners to-date.

File scans can take a long time – there are a lot of files and a lot of data to process. Shield schedules scans to run automatically in the background using the WordPress Cron subsystem.

We’ve engineered the scans so that they can run reliably in the background without interruption or breakage, event on resource-limited web hosting.

However, one area which we haven’t taken advantage of, until now, is background processing via WP-CLI.

Let’s examine how scans are run normally…

In order to process an entire scan, all the scan items are split up into smaller chunks that are each processed in 1 web request. When one chunk scan is complete, it triggers a web request back to the site to process the next chunk in the queue. Each time this happens, WordPress must be re-initialised from scratch. This has a cost, and if we can avoid this, we should.

Now consider WP-CLI, which is essentially “WordPress on the command-line”. Since it’s not a “web request”, many of the limitations that might be present on a web request no longer apply.

One of these common restriction is an “PHP execution time limit”.

Most web hosts enforce this time limit and it’s usually somewhere between 15 and 30 seconds. It means that the instant a PHP web request exceeds the time limit, it’s killed by the hosting server. It doesn’t shutdown cleanly, it’s just killed immediately.

This isn’t good.

We avoid this from happening with our scans by using the chunking method described earlier. But imagine a scenario where you don’t have the time limit? You could just process large tasks all at once.

And this is one of the advantages that WP-CLI offers us.

With our enhancements to the ShieldPRO scanning system, we can now more easily integrate with WP-CLI. In fact, you can now trigger new, on-demand scans directly from WP-CLI itself.

Also, if you’ve disabled your built-in WP Cron and prefer to run it using WP-CLI, Shield will detect this and take advantage of it in order to run the scans in a single go without chunking.

Many More Scanning Improvements

All improvements to the scans are based around the 3 key areas outlined above. In the section below we’ll get into some of the specifics of the improvements we’ve added.

#1 File Diffs To See What’s Changed

When Shield discovers a file has been altered, the first thing we want to know is: “What’s changed?”

Answering this can be a huge challenge in and of itself. But not any longer, with ShieldPRO 13. Wherever possible, Shield can now display a fully details file diff to show you exactly what’s changed throughout the file

See the video below.

#2 File History To See How Results Have Been Managed Over Time

Until now Shield only stored the results of the latest scan and nearly anything that came before that scan was lost.

With our new data storage models we can now store historical results and take these into account when displaying the latest results.

You’ll now be able to view a history of a particular file result, and see how it has changed or been repaired over time.

#3 Simplified Scan Options

We’ve changed the file scanning configuration options significantly, removing several options altogether and replace them with a single, unified option to enable or disable automatic file scanning.

We’ve also removed the old option to automatically ignore certain unrecognised files. This option has been replaced with the newer “Scan Exclusions” option that is more versatile. You may need to update this to reflect any changes you’ve made in the past.

There are plans for future enhancements to the scan results screens where you’ll be able to ignore certain files/paths based on the results on display. This will come a bit later, however.

#4 Enhanced Fingerprints For Premium Plugins and Themes

We’ve been building our knowledgebase of premium plugins and themes through our ShieldNET API infrastructure. With a few more tweaks we’ve made significant improvements to the API and its ability to provide file hashes for premium WordPres plugins and themes.

We’re the only WordPress Security provider than can provide file hashes (or fingerprints) for premium plugins. For example, at the time of writing, Yoast SEO for WordPress Premium is at version 17.5.

Other ShieldPRO 13 Enhancements

As with each major release there are alway improvements made in the background to enhance performance and fix bugs and inconsistencies. In this release, the following improvements were made (among others):

WP-CLI Traffic Log Capture

If you use WP-CLI to control and manage your WordPress, you may want to be able to see what the full WP-CLI request was in the audit trail. This is similar to seeing what the web request was. This is now available in ShieldPRO 13.0.

Fixes for Yubikey Registration

In some cases adding and removing yubikeys for One-Time Passswords (OTP) to your WordPress user profile was unreliable. We discovered a race-condition when submitting the profile details that would often interrupt the process. This is now fixed.

We’ve also added some Yubikey OTP verification when adding a Yubikey device. We’ve made it impossible to register a Yubikey device on your WordPress profile without it first verifying that the OTP sent from the device is valid.

Fixes for IP Address Management

A bug was discovered where unused IP addresses weren’t being properly removed from the database during cleanup.

We also made the IP Analyse tool more performant by making the IP Select/Search tool 100% dynamic. This makes IP lookups for large datasets easier to use without breaking performance and usability on the browser.

Comments, Feedback and Suggestions

We’re always working to improve Shield Security by adding new features and enhancing existing ones. As always, we appreciate your feedback on areas to improve or items you feel would help make your job as a WordPress Security administrator a bit easier.

Feel free to leave your comments and suggestions below.