Work at hand
Apart from the PHP issues, there are a few other things under construction: A new homepage, and a suite to process network-related logfiles.
For the new homepage I plan to use Mark Daniel’s VmsWasdContentManagementSystem – a native VMS executable that can handle this type of posts – even blogging is an option (perhaps, any blog on this site may be redesigned using this package). I had the beta installed, so I removed it to prevent problems that coud arise; downloaded the latest version, (both the sources and the AXP objects), built and installed it. It does require some configuration, and mapping in WASD, and to get famliar with it (and because of the recommendation) I set up the example as in the documentation. But either I don’t understand or mis-interpret the docs, or these are inconclusive (incomplete of plein wrong – I cannot tell), I ended up with a message:
ERROR 403 - reported by VWcms
Site directory not configured!
To be investigated….
Network logging
It’s an idea for quite some time: Scan all incoming network access, find out who’s attempting to hack, or abuse the systems, and shut the door for these people.
I started today with a program to scan the SYSLOGD logfiles on Diana: the firewall on the dge of the domain logs all access in this file, and when it is over 25.000 blocks in size, it’s cycled, and all cyccled files are stored in a zip file during the monthly maintenance process. Other files to process are the PMAS and FTP logfiles, and the access logs of the webserver.
So I need a program to convert these files into data that can be stored and analyzed, and that is also capable of updating the firewall with the top-100 addresses; the Vigor is capable of storing 192 single addresses, address ranges or networks that can be denied access – at the gate.
I started with a DCL-procedure that splits the SYSLOGD output – either active or archived – into incoming and outgoing traffic; each of which is next split into protocol-specific files; so at that moment, I have all lines of logging for every protocol, either incoming or outgoing – in exactly the same, fixed format. Therfore, it’s very easy to extract the required data from these files: date and time of access, the source and destination address and port – and the protocol.
Since there is quite a number of archives to process, I also created a procedure to scan a directory for these files – put there by hand of by unzipping an archive – and have each file processed that way. I’ve taken a decision to mark each final output file by the date it is created, and once created (if not existing) it will be extended with each SYSLOGD file that is processed.
This works fine now – next is the extraction of the same data from the PMAS logfiles, but IIRC, that has been done already, I just have to look fro them; otherwise, it is not a lot of work to do the same for these files. The same applies to the web-server access logfiles: Create a procedure that can handle one, and I’m done (just add a wrapper that passes the filename of the file to be processed.).
And, of course, a program to store this data into a database, a program to analyze the data, and one to update the firewall accoringly.
License!!!
A few days ago, I found out – by accident – that the PMAS license expires tomorrow. I sent a request for a new license to the address I know exsists for that type of message – but it bounced. Next, I sent it to the address of Hunter Goatley – who’s in charge of the hobbyist licenses – and that bounced as well. So I sent it to the support desk of Process Software, but since I have a free license, they couldn’t help me; in stead they passed another address – which bounced also, so I was advised to contact Hunter directly – which didn’t bounce for the next hour. So it is likely to arrive; hopefully Hunter is not on holiday, and the license arrives is time – or I’ll be buried under all the messages that PMAS is now blocking ro rejecting…Fingers crossed….
27-Oct-2011
Wiki abuse
The number of fake users – those with undoubably fake e-mail addresses – that create an account on the VMS wiki, just to add text with inappropiate content, linking to obscure sites and so on, has suddenly increased. Every few days, there is yet another account that abuses the wiki. Luckiliy, the system has been configured to notify me for new users, and so it’s easy to track them down, de-spam the wiki from their posts and disable the accounts. But it would be nice if the software creating users would check the address to exist: send a confirmation mail to that address and finishing the registration after the address has been verified; next, no posts but their personal page can be done, and only aftre that is finished, they can contribute….
Of course, since MoinMoin is an open-source application, it’s possible to create such a method. But it requires knowledge of Python, and insight in the code. I don’t know Python, though I could learnm to use it; but it requires time, and there are other, more important actions to be taken. And this is not occurring that much that I have to spend a lot of time removing these abusive users and their posts. Just annoying, at this moment…
Spam statistics
I took a look to PMAS’s reports tonight, scanning the statistics for the years I’ve been using this product for filtering incoming mail. These reports clearly show the increase of spam until 2009, in 2010 there was less, and it looks as the amount of spam decreases even more this year. But I need to do a more thorough investigation using all logs – splitting it up in domain, address, if possible, and many other things, store the results in a database and run statistics on that. That would also include rejected messages (that seem to be excluded from the reports) and relay attempts (not shown either). Again, this takes time, and should be part of a larger project for forensics. Again, a project to be started….
Projects
Speaking of projects:
* Remove MySQL and use another database. This requires a drastic change to the WordPress code, and I don’t think such a change would make it into the community. I’m looking around for an alternative: Mimer or Ingress, or even Caché might do.
* Remove PHP alltogether and move to a native program that I can use. Mark Daniel – the creator of the WASD server – has created a content management system that I could use – either as-is, or as a base for own development.
* Remove Python – and the wiki – for the mentioned reasons. Here this native CMS coiuld be used as well. But here as well, I could opt for yet another solution.
* Redo the home page. I could use this product as well, or write my own, fully from scratch or based of the CMS….
Main reason is that MySQL, PHP and Python are resource-hungry. PHP is anyway, it’s an interpreter; Python does some compilation and so it’s faster once started. I could also change the configuration for the sake of speed but there I would suffer decreation of performance elsewhere. The systems holds 512 Mb of memory, and adding more is – due to it’s age – very expensive. I think it a bit odd that older types of memory are far more expensive than state-of-the-art chips…
Second, since Oracle bought the company that is responsible of MySQL, and the way the Oracle company works, it’s my idea that the devlopment of MySQL is now under full control of Oracle. That;s the reason why there will be no MySQL-6 on Itanium. The community is pushed aside, it’s Larry Ellerson – or any subordinate – that dictates the way the database is developed. To me, that’s enough to abandon MySQL as soon as I can. Luckily, there are alternatives: Ingres, as an open source (though I may have to build it from source, the openVMS version is not freely available, alas); caché and Mimer can also be considered, and, once the product has been re-ported to OpenVMS, Postgress is as well (Knowing that Ingres was first (and still is) developed on OpenVMS and Postgres has been derived from Ingres – and once ran op openVMS as well, it might be not too much of a problem. Rumours are that the Postgress libaries are available)