PHP Notices

Since Zymic’s PHP is set to display errors of the level E_NOTICE, users often find they are getting errors on Zymic that they are not getting on their local test server or previous host.  E_NOTICE level errors are good practice recommendations that point out where you’ve written valid but improper PHP.  It is recommended that rather than suppress E_NOTICE level warnings you display them during the development process and fix your code so that the errors are never generated.  This improves the quality and reliability of your code.

If your local test server does not display notices then I would recommend you change your local PHP configuration to start generating them.  You can do this by modifying the “error_reporting” directive in your php.ini file.  In a system where notices are disabled it is typically set up as this:

error_reporting = E_ALL & ~E_NOTICE

This format takes E_ALL, a group of many error levels including notices, and then removes notices from the list by adding ~E_NOTICE (not E_NOTICE).  So you can enable E_NOTICE level errors by making it simply:

error_reporting = E_ALL

The next few topics are about some specific E_NOTICE level errors you may have found and what they mean then how to resolve them.

PHP Notice: Undefined Index or Undefined Variable

This notice refers to using a variable or a member of an array that has not been defined prior to its use.  The simplest example of this is to take this code:

1
2
3

<?php
   echo $foo; // At no point have we defined $foo, so using it is probably bad
?>

This will produce an error along the lines of “PHP Notice: Undefined variable: foo in __FILE__ on __LINE__”. This is obviously a very trivial example and it rarely comes that simple in real code, but it is an illustration of what triggers this error. Attempting to reference some data that doesn’t exist can easily lead to unexpected behaviour, and it’s fairly obvious that all variables/array members you try to read from should be defined and initialised before you try that read.

A common context where “Undefined Index” will turn up is in people handling the superglobals like $_POST. If you attempt to use $_POST['foo'] without checking some input with the name “foo” has been posted to the script will trigger this notice. So, that’s what the notice is and what causes it, but how can you prevent it from appearing? Well, there are some tests that PHP has which can check the data, and then you can make sure that you’re only attempting to read the data if the variable/array member is set or is not empty. So say you want something that outputs “Hello, {name}” to the browser and it works on GET data, an implementation which will trigger this notice might be:

1
2
3

<?php
   echo 'Hello, ', htmlspecialchars($_GET['name'], ENT_QUOTES);
?>

But this script doesn’t work particularly well when a name isn’t provided, you will get a notice and the output will just be “Hello, “, which is not particularly meaningful – we can do better than that. So if we add in a check using PHP’s empty() function to see if there is some data provided we could instead write:

1
2
3
4
5
6
7

<?php
   echo 'Hello, ';
   if(empty($_GET['name']))
      echo 'Stranger';
   else
      echo htmlspecialchars($_GET['name'], ENT_QUOTES);
?>

This is very similar to the last, but it will not trigger a notice if there is no name provided to the script, and in fact handles it by instead outputting “Hello, Stranger” if no name is provided. This is a bit more elegant, and there are many ways you could tackle this. You could have the case where if no input is provided we show a form instead for the user to provide a name.

PHP Notice: Use of undefined constant

This one is very much similar to the last one, but is worth mentioning separately because there’s enough to be said about the mistakes that lead to it, and how you can make these mistakes and not realise it without the benefit of notices. This notice is about the use of undefined constants in code, the interesting thing is how PHP handles a case where an undefined constant is found. It will trigger this notice, but since this is valid PHP it will take a value for the constant for the script to use, and that is the identifier (or name) of the constant, thus an undefined constant foo will evaluate to the string ‘foo’.

Because of this fact you often see cases where an array member is referenced via $array[foo] rather than $array['foo']. If notices are disabled, then it is possible that the author of that code will not notice the mistake (though syntax highlighting in your editor should mitigate this somewhat) because while foo is an undefined constant the two are equivalent. However, this is not always the case. An easy example to demonstrate this is this. Take file1.php to be this:

1
2
3
4
5

<?php
   $array = array( 'foo' => 'Hello there', 'bar' => 'Goodbye');
   echo $array[foo]; // Note here we've not quoted foo so it's a constant,
                     // and in the context of just this script it's undefined.
?>

When you visit this page you will find it triggers this notice and outputs “Hello there”, however if we were to also have a file2.php containing this:

1
2
3
4
5

<?php
   define('foo','bar'); // define the constant foo with the value "bar"
   require 'file1.php'; // and then bring in the file we just wrote,
                        // with this constant in scope
?>

When you visit file2.php you’ll find the notice has gone and we’re now presented with “Goodbye”, this is the crux of the issue, when you unintentionally use an undefined constant you no longer have any idea how that script will run when it is part of a larger program. What it will actually do is now undefined behaviour. So, if what you mean to write is a string, make sure it has quotes around it, so the PHP parser knows that it’s a string – not a constant.

That’s it… for now

I may add to this at a later date to add in anything else that gets asked frequently, we shall see.

The website was specified to be a very simple, mostly static collection of pages including an easy to use blog system for comments the company’s primary consultant had on issues related to Systems Engineering.  I designed a simple interface and translated that into a working website which Arbutus can use to advertise themselves to potential clients.

If you’re interested then have a look at what I came up with for Arbutus Technical Consulting.

So, what’s the new name for the software?  Well, that’s possibly still in flux, but for the moment I’ve decided I might go with polymer.  A slightly nerdy nod to the fact I want to make this release inherently extensible, as much as is needed.

Polymerisation

Of course, with a name like that I really should elaborate on just why it’s going to be more modular and easily extensible than the previous bot.

There was nothing really wrong with the implementation before, and much of it has been kept constant in the new implementation.  The most notable changes are in the format used to write modules, which has been simplified somewhat – and the fact that module interaction will be made possible allowing modules to reuse functionality included in a module that is already loaded into the bot.

My current WIP draft for module layout is this:

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63

<?php
 
// module description up here?
//
// that would make sense.
 
class mod_example
{
   // any local variables required
   private $localvar;
   private $othervar;
 
   /// Core and required methods:
   // init, performs any initialisation required
   public function init()
   {
      // initialisation stuff... for example:
      $this->loadConfig(); // <-- (re)loads configuration in {confdir}/{name}.conf
 
      // the triggers and hooks
      registerTrigger('ping','respond');
      registerHook('passive');
   }
 
   // a rehash method which handles a complete config reload
   public function rehash()
   {
      // rejig internals in case our config has changed
      $this->loadConfig(); // <-- like this again perhaps?
   }
 
   /// about() and help() will probably make an appearance though since some are
   /// internal they need not include them.
 
   /// Trigger/Hook implementation
   // Here's a trigger, triggered of course by a !ping command.
   public function respond()
   {
      // note the lack of any arguments, instead the information will automatically
      // be made available through methods/variables contained within the base
      // class.  This simplifies format, and allows us to make triggers and hooks
      // constant.
      $target = $this->state->target;
      $nick   = $this->state->caller;
      // module intercommunication:
      if( is_object($_irc) )
         $_irc->msg($target,$nick.', pong');
   }
 
   // And here's a hook, called on every new packet
   public function passive()
   {
      // do stuff
   }
 
   /// And as always, you can declare internal functions for personal use.
   private function helper($arg1,$arg2)
   {
      // do something helpful
   }
}
 
?>

This format may undergo changes as I work on it, but it’s likely to look something like the above when I’m done.

Any suggestions/comments are very welcome, since this is going to be the interface anyone who wants to write modules will be using, so it’s important that it’s suitably intuitive.

As far as I’m aware PHP doesn’t contain anything at all for controlling threads, however it does include some process control functions which allows us to use a multi-process model for concurrent programming.

Introducing Process Control

The functions we need come via the Process Control module in PHP.  This is not included by default in the PHP Apache module (because it’s more complicated in such cases), if you want to use it for a website you will need to use PHP as a CGI module or compile mod_php with –enable-pcntl, in this case I’ll be testing using the PHP CLI binary, in which it’s included by default.

The key function that this module provides for us is pcntl_fork().  This works much like the fork you may have come across in other languages, it creates a clone of the current process which then computes a separate code path to the parent that called it.  The important thing of course being that the clone (or child) process does its computation while the parent carries on with whatever it was doing.  This is key to parallelising our PHP applications, and allows us to create responsive applications which can still do lengthy tasks.

Process Control in Use

So where might this be useful?  A trivial example would be when you want to perform a task that takes a number of seconds, and you want the user to receive output from the program while it’s being computed.  A simple example of this could be the code below:

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38

<?php
 
// this is a place-holder for a hypothetical function
// which takes a long time to compute, let's say the
// eventual answer would be 5...
function foo()
{
   sleep(10);
   return 5;
}
 
file_put_contents('tmp',0); // we'll also assume we know the answer isn't 0.
 
// Here's the fork, the process ID will be stored in $pid for the main
// program, and it'll be 0 for the child process.
$pid = pcntl_fork();
if($pid == -1)
   die('Fork failed'); // Our fork failed, thus our program did.
elseif($pid == 0)
{
   // here's our forked process
   file_put_contents('tmp',foo());
   echo "Done\n";
   exit;
}
 
// In the meantime we'll keep our parent process looping.
$timestamp = time();
while(file_get_contents('tmp') == 0)
{
   if($timestamp != time())    // once a second...
      echo 'Calculating...',"\n"; // give some output
   $timestamp = time();
}
 
echo file_get_contents('tmp')."\n"; // output our answer from the fork.
 
?>

As you see we have simply made a test-case here, there’s a function with a sleep(10); which takes the place of any long-running function, and while it is running we inform the user that the task is underway, once a second until it is complete.  It’s quite a trivial example, but it shows that both the echo statements and the sleep were functioning in parallel when viewing the output, which is predictably as follows:

bash-3.1$ php -f test.php
Calculating...
Calculating...
Calculating...
Calculating...
Calculating...
Calculating...
Calculating...
Calculating...
Calculating...
Done
5

Another example might be if you were performing a repetitive task on many targets, such as all the files in a directory.  You could loop across the directory forking off a new process which would handle each file, then back in the main loop you simply wait for all of the processes to finish and then wrap up any loose ends you might have.

Conclusion

So there you go, process control.  A useful tool if you need to speed up a parallelisable program/algorithm.  Of course, in most cases speed isn’t too necessary in PHP applications, so the practical applications of process control aren’t very numerous.  In cases where speed is not important, it is usually better to keep the application simple, since it means it is more likely to be correct.

However, if you are writing something real-time, but don’t want to bog down your main application loop – making your application unresponsive, process control comes into its own and will save the day.

Concurrency in zBot

As I mentioned in the previous post, concurrency is something I’ve been meaning to include in zbot for a while now. zbot is a real-time PHP application, and as an IRC bot, a response time of over a second or two is starting to seem sluggish. Sometimes however some responses can’t be given in this length of time because they rely on an external site, and so the bot must wait for a response before it can reply. In such cases I would like the main loop to be free to continue watching for more easily serviced queries. So that it could reply to those first while it is waiting.

It also allows me to safely parcel off largish tasks without having to worry that the bot will ping out, as the core will continue looking for PING packets. For example, I would like to implement a news plugin which would download five or more RSS feeds and then parse them to check for new news stories. This would be ideal for process control, as I could fork off another process which would handle the downloads and parsing while the main bot continues to function.

For this purpose I have written a taskScheduler class to which I can send jobs which are then evaluated in a forked process. The class keeps track of all scheduled jobs, and performs a callback when the job is completed. This might prove to be a useful separate class, so I’ll likely both include it in zbot, and separately on my site with some documentation when it’s finished.

Identifying the problem(s)

Building a new 2.0 version of my IRC bot has been something I’ve wanted to do for a while now, and there are a few main reasons why I’ve decided that it should be a fresh start rather than an update.  These are mostly (fairly) fundamental feature issues, which should be implemented quite a long way down the source tree – and so modifying the source to patch them in would be a tedious and lengthy process which might introduce a fair few bugs into the codebase.

The main things I want to improve in the new version are as follows:

• Generally cleaner code (I might choose to release this source publicly, and I want it to be nice).
• Improve the module system to allow the modules to intercommunicate, by doing this I can abstract even some of the “core” functionality into modules, which will allow for patches at a later date without a restart.
• Ground-up design for a bot rehash feature, in which it reloads all settings from configuration files.
• Simpler text-file configuration files which will be parsed and read-in.
• Work out a system for concurrency, perhaps spawning child PHP processes to handle any tasks which might take a long time (such as fetching files online), this would free up the rest of the core loop to handle other input while these tasks are performed.
• Additional bot features, such as automatic tracking of users in channels, dynamic topic changes (so for example, we could specify a field which would be changed by an external source, allowing us to do things like include the current stable version number of PHP in #php).
• Web interfaces for some bot data, like current factoids, quotes, and possibly a module repository that people could submit to.

Moving forward

I hope to look at building a new version soon, sorting some groundwork for things at the moment, I’ll try to keep this updated as work goes on.

Of course, my good buddy Ed (Bread) will probably assist as he did in the last version (he provided the initial module system and a few plugins), and maybe this time we’ll actually write some documentation so people can write their own modules easily! :p

Since I will be hosting this on my shiny new VPS, we have more freedom (having a web interface was not possible before), and it means we can have one central source tree (I’m thinking stable/ and testing/ with a script to update stable when testing is shown not to break things ).