20-Oct-2013

Mail in error
For some reason, NOT ANY message has been received for over a day – not even quarantined or discarded. This is pretty weird, so I took a look and found that PMAS has gone into a “DNS-blacklist ALL” mode. Mail sent from my GMAIL account – that normally would arrive – was blocked as well. Even when I explicitly allowed al mail from gmail.com, of any account from that domain, mail sent from gmail was blocked:

Delivery to the following recipient failed permanently:

(my address)

Technical details of permanent failure:
Google tried to deliver your message, but it was rejected by the server for the recipient domain grootersnet.nl by mail.grootersnet.nl. [82.161.236.244].

The error that the other server returned was:
554 5.7.0 Address (209.85.223.196) blacklisted (2)

(this was after I disabled the first entry in the configuration).

In order to receive mail – even spam that would otherwise be rejected, discarded or quarantined – I disabled PMAS by opening port 25 and remove the forwarding on port 25 to port 2525 – the one that PMAS listens on. Now mail arrived so it definitively was a PMAS issue. But what cuased it could not be determined.
A few days ago, I downloaded that latest version (PMAS032-050) from Process; I went there after I found I couldn’t create reports for this year and Hunter gracefully admitted he made a mistake and set a new file available – and with this access, I also retrieved this latest version.
I installed it in the right location, moved files (configuration, spam database, log files and statistics database) and restarted PMAS. Now al seems to be in working order again. Just have to copy what’s been quarantined and discarded.
Throttle redefined
The problems I encountered a few days ago: overload to PHP_WASD processes, made Mark Daniel propose another setting. So I have taken some precautions, so the amount of accesses to the blogs is now limited, and hopefully wide enough for normal use, and tight enough to prevent system exhaustion. I’ll monitor this for a few days: you may encounter 503-errrors: stating the service is not available, or some limit is reached. Big abusers may be locked out on a more permanent basis: I now know how to do that 🙂

12-May-2013

Memory leak?
In time, usage of virtual memory increases from less than 25% to about 60. All in all, it took 140 days so it’s not that dramatic. Nevertheless, it’s something that should not occur; it means that some process in the system requires more and more memory and doesn’t return it. This might be legitimate, but it could also reveal an error. It must be a process that is continuously running, and that rules out PHP, because these run idle for a very limited time. I can also rule out VMS itself, because that has been tested on this amount of memory leakes. Leaves the webserver itself, MySQL and the spam filter.
WASD is unlikely, I know it’s memory requirements are limited, memory is allocated once (on start of the server) and all worker processes on my system run as long as there is something to do – and they disappear when idle for some time, so allocated memory is returned automatically (that’s the way VMS is built 🙂 ).
MySQL could be a cause; it keeps record of changes and caches results. So I stopped and restarted the MySQL server, memory usage dropped somewhat but just about 5%.
Next culprit is the mail filter, and indeed, stopping and starting these processes caused the memory use to drop below 25:

after this action was completed.
There is something to look into.
I have observed that the increment of memory usage occurs in steps, that coincide with high CPU requirements, a increment of number of processes, and, by that, increased paging (in fact: usage of real memory drops dramatically, at the same time, usage of the page file increases. This means that modified pages are written to disk….). If this coincides with the heavy mail usage found in the logs (operator, PMAS and router) this means there is something for Process to look into. Such an event may start a lot of worker processes, that exit when idle for some time. Memory should be returned in that case, but it looks as if memory, once allocated, is kept allocated. It might be legitimate, since the memory recommendation for PMAS is 1 Gb minimal and Diana has only half of that. But if this coincidence exists, it may be worthwhile to note this to Process. Better be sure….

14-Aug-2012

Work at hand
Apart from the PHP issues, there are a few other things under construction: A new homepage, and a suite to process network-related logfiles.
For the new homepage I plan to use Mark Daniel’s VmsWasdContentManagementSystem – a native VMS executable that can handle this type of posts – even blogging is an option (perhaps, any blog on this site may be redesigned using this package). I had the beta installed, so I removed it to prevent problems that coud arise; downloaded the latest version, (both the sources and the AXP objects), built and installed it. It does require some configuration, and mapping in WASD, and to get famliar with it (and because of the recommendation) I set up the example as in the documentation. But either I don’t understand or mis-interpret the docs, or these are inconclusive (incomplete of plein wrong – I cannot tell), I ended up with a message:
ERROR 403 -  reported by VWcms
Site directory not configured!

To be investigated….
Network logging
It’s an idea for quite some time: Scan all incoming network access, find out who’s attempting to hack, or abuse the systems, and shut the door for these people.
I started today with a program to scan the SYSLOGD logfiles on Diana: the firewall on the dge of the domain logs all access in this file, and when it is over 25.000 blocks in size, it’s cycled, and all cyccled files are stored in a zip file during the monthly maintenance process. Other files to process are the PMAS and FTP logfiles, and the access logs of the webserver.
So I need a program to convert these files into data that can be stored and analyzed, and that is also capable of updating the firewall with the top-100 addresses; the Vigor is capable of storing 192 single addresses, address ranges or networks that can be denied access – at the gate.
I started with a DCL-procedure that splits the SYSLOGD output – either active or archived – into incoming and outgoing traffic; each of which is next split into protocol-specific files; so at that moment, I have all lines of logging for every protocol, either incoming or outgoing – in exactly the same, fixed format. Therfore, it’s very easy to extract the required data from these files: date and time of access, the source and destination address and port – and the protocol.
Since there is quite a number of archives to process, I also created a procedure to scan a directory for these files – put there by hand of by unzipping an archive – and have each file processed that way. I’ve taken a decision to mark each final output file by the date it is created, and once created (if not existing) it will be extended with each SYSLOGD file that is processed.
This works fine now – next is the extraction of the same data from the PMAS logfiles, but IIRC, that has been done already, I just have to look fro them; otherwise, it is not a lot of work to do the same for these files. The same applies to the web-server access logfiles: Create a procedure that can handle one, and I’m done (just add a wrapper that passes the filename of the file to be processed.).
And, of course, a program to store this data into a database, a program to analyze the data, and one to update the firewall accoringly.
License!!!
A few days ago, I found out – by accident – that the PMAS license expires tomorrow. I sent a request for a new license to the address I know exsists for that type of message – but it bounced. Next, I sent it to the address of Hunter Goatley – who’s in charge of the hobbyist licenses – and that bounced as well. So I sent it to the support desk of Process Software, but since I have a free license, they couldn’t help me; in stead they passed another address – which bounced also, so I was advised to contact Hunter directly – which didn’t bounce for the next hour. So it is likely to arrive; hopefully Hunter is not on holiday, and the license arrives is time – or I’ll be buried under all the messages that PMAS is now blocking ro rejecting…Fingers crossed….

10-Mar-2010

Spam source(s) unwired ?
yesterday’s surprise:
61 messages have been received yesterday; 11 have been delivered and 7 discarded. None has been quarantained and the rest (53) has been rejected anyway because of their presence in a blacklist, or because their score was way too high (> 200). Most numbers are rather normal, but none quarantained is remarkable.

Router issues
The new router allows two WAN connections, but the second one shares it’s outlet with one of the LAN ports. Here, I ran into a problem: when the printer is connected to this port, a static route is added to the route-table, and this causes a problem when a system on the LAN tries to access the internet – but existing connections seem to have no problem at all). This behaviour is shown in one of the diagnostic screens. The situation can only be reversed by removing the physical connection and reboot the router.
This has been communicated to Draytek, they have looked into it but said it should just work. But it doesn’t – and I tested it again, taking screendumps from what I’ve seen and sent it over to them. Now it’s just a matter of waiting on their reply. The problem is that I cannot tell – from the LAN – whether incoming traffic is also effected. It may well be, and therefore I cannot risk being locked out for some time.