Posted on

An honest hacker

Found this in operator.log:

%%%%%%%%%%% OPCOM 29-DEC-2007 21:23:47.65 %%%%%%%%%%%
Message from user TCPIP$FTP on DIANA
User Name: anonymous
Source: goldzulu.takethishost.net
Status: NOPRIV -- File access violation
Object: WEB_DISK2:[public.anonymous.test]

FTP log shows he made just one attempt and left:

%TCPIP-I-FTP_SESCON, FTP SERVER: session connection from goldzulu.takethishost.net at 29-DEC-2007 21:23:46.40
%TCPIP-I-FTP_NODE, client host name: goldzulu.takethishost.net
%TCPIP-I-FTP_USER, user name: anonymous
%TCPIP-I-FTP_OBJ, object: WEB_DISK2:[public.anonymous.test]
%TCPIP-I-FTP_CHINFO, TCPIP$FTPC00036: Failed to create directory
%SYSTEM-F-NOPRIV, insufficient privilege or object protection violation
%TCPIP-I-FTP_USER, user name: anonymous
%TCPIP-I-FTP_SESDCN, FTP SERVER: session disconnection from goldzulu.takethishost.net at 29-DEC-2007 21:23:47.81

Mind the domain name: I would not expect such honesty on the intentions from a malicious user!
Believe it or not: the node and domain leads to an address in the US, and dig gave this info on the host:

$ dig goldzulu.takethishost.net

; < <>> DiG 9.3.1 < <>> goldzulu.takethishost.net
;; global options: printcmd
;; Got answer:
;; ->>HEADER< <- opcode: QUERY, status: NOERROR, id: 54539 ;; flags: qr rd ra; QUERY: 1, ANSWER: 1, AUTHORITY: 3, ADDITIONAL: 1 ;; QUESTION SECTION: ;goldzulu.takethishost.net. IN A ;; ANSWER SECTION: goldzulu.takethishost.net. 900 IN A 66.98.228.61 ;; AUTHORITY SECTION: takethishost.net. 14400 IN NS ns1.takethishost.net. takethishost.net. 14400 IN NS ns2.takethishost.net. takethishost.net. 14400 IN NS ns3.takethishost.net. ;; ADDITIONAL SECTION: ns3.takethishost.net. 14400 IN A 209.85.25.142 ;; Query time: 796 msec ;; SERVER: 127.0.0.1#53(127.0.0.1) ;; WHEN: Mon Dec 31 16:23:23 2007 ;; MSG SIZE rcvd: 129

and on the address:

$ dig -x 66.98.228.61

; < <>> DiG 9.3.1 < <>> -x 66.98.228.61
;; global options: printcmd
;; Got answer:
;; ->>HEADER< <- opcode: QUERY, status: NOERROR, id: 8305 ;; flags: qr rd ra; QUERY: 1, ANSWER: 1, AUTHORITY: 2, ADDITIONAL: 2 ;; QUESTION SECTION: ;61.228.98.66.in-addr.arpa. IN PTR ;; ANSWER SECTION: 61.228.98.66.in-addr.arpa. 86400 IN PTR goldzulu.takethishost.net. ;; AUTHORITY SECTION: 228.98.66.in-addr.arpa. 259200 IN NS ns1.ev1servers.net. 228.98.66.in-addr.arpa. 259200 IN NS ns2.ev1servers.net. ;; ADDITIONAL SECTION: ns1.ev1servers.net. 172800 IN A 207.218.245.135 ns2.ev1servers.net. 172800 IN A 207.218.247.135 ;; Query time: 2718 msec ;; SERVER: 127.0.0.1#53(127.0.0.1) ;; WHEN: Mon Dec 31 16:20:18 2007 ;; MSG SIZE rcvd: 161

WHOIS gave on the domain:

Whois Server Version 1.3

Domain names in the .com and .net domains can now be registered
with many different competing registrars. Go to http://www.internic.net
for detailed information.

Domain Name: TAKETHISHOST.NET
Registrar: TUCOWS INC.
Whois Server: whois.tucows.com
Referral URL: http://domainhelp.opensrs.net
Name Server: NS1.TAKETHISHOST.NET
Name Server: NS2.TAKETHISHOST.NET
Status: ok
Updated Date: 03-jan-2007
Creation Date: 15-jan-2004
Expiration Date: 15-jan-2008

so it will expire within a few weeks. Probably hijacked? or deliberately setup for the porpose some time ago? Who knows..

Posted on

28-Dec-2007

Give the processor some work to do
I created certificates to be used to run the protected sites over SSL, so port 443 in stead of 80. There are some issues with it – since all sites share the same IP address, I must use one shared certificate for all: *.grootersnet.nl as target.
The basic drive to get this working was a request to access the researcher’s genealogy website. A new request for access has been granted and i thought it a good moment to update the site, and install a different access scheme – including usage of SSL.
The same applies to the operator desk, at least part of it should be protected (all, for now) and obviously webmail.
The latter proved to be most problematic. A nasty error “Object protection violation” messaeg shows up, even after reversing the change. It boiled down tom an error in the configuration file, where a specific entry allowed access without login – hence REMOTE_USER remained empty, causing this error. It took a few hours to get it all properly working but in the end, all runs smoothly – taking a lot more time to finish.

Posted on

27-Dec-2007

Native dynamic content
Work is progessing. The basic stuff (the old VAX modules) works, using the test program and definitions, but the HTML output stalls in writing a bunch of lines – the very same functionality that works in the test program. So this must be something minor. To be solved shortly 🙂 The first thing to do is the main page of the site, creating a more vivid appearance. To strat with data stored in RMS files – no real database to start with, but it might well do in some future development. RMS is fine for now.
Crashes
seem to have occurred on the anti-spam working proces (0002 stopped by ACCVIO, 0008 started) and MySQLServer when accessing a newly added entry. It has been started by the watcher program – nice!
There have been several problems with the PHP engine, all the same:
%HTTPD-W-NOTICED, 25-DEC-2007 22:31:49, CGI:1969, not a strict CGI response
-NOTICED-I-SERVICE, http://www.grootersnet.nl:80
-NOTICED-I-CLIENT, aaa.bbb.ccc.ddd
-NOTICED-I-URI, GET (33 bytes) /sysblog/index.php?p=161#comments
-NOTICED-I-SCRIPT, /sysblog/index.php sysblog:[000000]index.php (cgi_exe:phpwasd.exe) SYSBLOG:[000000]index.php
-NOTICED-I-CGI, 254445425547424F4F542D572D43484E2C2061737369676E (62 bytes) %DEBUGBOOT-W-CHN, assign channel system service request failed
-NOTICED-I-RXTX, err:0/0 raw:235/0 net:235/0
This is a nasty habit – a known issue within the PHP engine. The only thing I can do is recommend to retry – tomorrow…
Thete is a new patch for DECC – the C RunTimeLibrary – required for CIFS – AKA Samba – perhaps it will solve these (and other) issues – like there exist in MySQL. Who knows these crashes are gone when installed ….

Posted on

19-Dec-2007

All seems right
Having done some (not many) blog and forum work, the only issue found so far is an error that shows up once in a while:
%HTTPD-W-NOTICED, 16-DEC-2007 18:02:13, CGI:1969, not a strict CGI response
-NOTICED-I-SERVICE, http://www.grootersnet.nl:80
-NOTICED-I-CLIENT, aaa.bbb.ccc.ddd
-NOTICED-I-URI, GET (21 bytes) /tracks/index.php?p=5
-NOTICED-I-SCRIPT, /tracks/index.php tracks:[000000]index.php (cgi_exe:phpwasd.exe) TRACKS:[000000]index.php
-NOTICED-I-CGI, 254445425547424F4F542D572D43484E2C2061737369676E (62 bytes) %DEBUGBOOT-W-CHN, assign channel system service request failed
-NOTICED-I-RXTX, err:0/0 raw:181/0 net:181/0
No MYSQL issues – so far.
It seems re-configuring the system – and non-paged dymnamic pool in particular – did solve a problem or two. Or did the latest pathes? So far, there have been no problems in this area – besides that gettting data takes some time. It might be an idea to reverse changes in sizing – when I get back to the box.
PAMS however, ran into some problem last week:
%%%%%%%%%%% OPCOM 17-DEC-2007 12:48:39.22 %%%%%%%%%%%
Message from user SYSTEM on DIANA
%PTSMTP-E-WORKERDIED, worker PTSMTP 0001 (20200147) terminated unexpectedly
-SYSTEM-F-ACCVIO, access violation, reason mask=00, virtual address=0000000000000000, PC=0000000000000000, PS=0000001B
-PTSMTP-I-WORKERCONN, while processing connection from 220.83.123.34,2231
I have to notify process.com – but will need to get to the logs first (forgot about that). It’s just one of the worker processes and a new one is automatically created so there has been no interruption of receiving mail but this particular message.
Another thing to ask for: sometimes messages get through and are filtered by the nromal SMTP engine: all stating the domain is unqualified or not resolvable (these all got in this month):
2-DEC-2007 01:02:57.63 UNRSLVMF kenneth79virginio80@12dailyguide.com
3-DEC-2007 14:03:42.68 UNRSLVMF jeromy618@falconlimos.com
4-DEC-2007 00:35:37.56 UNRSLVMF jrabbat@horizonfcb.co.ae
4-DEC-2007 20:23:20.42 UNRSLVMF q-jack.com@tattooedwhitetrash.com
5-DEC-2007 20:16:26.87 UNQUALMF Keill
6-DEC-2007 00:42:12.31 UNQUALMF Ketten
7-DEC-2007 02:17:50.92 UNRSLVMF support@paypal.inc.com
11-DEC-2007 00:39:06.91 UNQUALMF Kalchreit
11-DEC-2007 19:53:24.11 UNQUALMF Kindler
12-DEC-2007 01:39:52.27 UNRSLVMF info@PayPal.Inc.com
12-DEC-2007 03:45:36.41 UNRSLVMF akguvyxy@in-addr.arpa
12-DEC-2007 03:53:52.28 UNRSLVMF jramos@ranido.com
13-DEC-2007 04:30:34.05 UNRSLVMF jr@drillerssupply.com
15-DEC-2007 15:26:10.37 UNRSLVMF jr@cgafin.com.au
15-DEC-2007 22:52:33.36 UNRSLVMF spendsf7@inmover.com
17-DEC-2007 13:23:18.07 UNQUALMF Perde
18-DEC-2007 07:25:53.96 UNRSLVMF jr.tolerd@kecoindustries.com
18-DEC-2007 16:35:12.70 UNRSLVMF jquiaro@dinaut.com.ve
just a few compared to what has been refused in the past, but it would be nice if this type would be filtered, or rejected, as well. Perhaps Hunter can work this out.

Posted on

State of procedures

(Triggered by an article by Jim Duff).

My remark on the issue – a failing disk in s shadow set was unnoticed – is mainly a matter of inappropiate system monitoring. Disks are prone to error and MUST be monitored. At least: watched in regular intervals. Well, very in 6 moths IS regular – but I mean shorter ones. Jim published a procedure that can do a good job, but some other things need to be considered as well.

A fair example is backup jobs. As are monitoring scripts. Any other procedure that runs automatically and is’more or less critical.
It is often assumed these jobs have 2 states: they have run successfully, or they haven’t. In a lot of cases, if not most, the system manager is only notifies if something went wrong – and no message usually means the procedure finished succesfully. So everyone relies on the absence of a message assuming backup has finished corerectly – until the moment of truth arrrives and a disk needs to be restored – wand it was found that the backup was actually non-existing.
This is No Horror Story. It happens…..

Principle procedures have 3 states:

  • it has run and finsihed succesfully
  • it has run but failed
  • it has not run at all

    • The first two must be brought to the attenmtion of the ssytem manager – or responsible operator, if that is appropiate. As soon as possible, preferrably – but it largely depends on priority. In some cases, failure of backup is not that important. In other cases, immediate action is a requirement.
    One medium – often used, I guess – is mail, or paging. In case of success, the message does not have to be a full body. In case of failure however, it might be required that more information on the error is added. In case of e-mail, the logfile, for instance.
    The third state is ‘delivered’ by absence of such a message. No message means Big Trouble.
    You can stretch this even further: signal each event separately. But it largely depends on the significance of each step – and the importance of the whole process.
    A watchdog program or procedure could add a monitoring facility – but mostly adds just one element in the chain. If that one brakes, it breaks monitoring totally, and you, as a system manager, will have no idea what’s gone wrong and right, and you will have to revert to the logfiles and other traces of activity (accounting, audit).

    The same applies to procedures mionitoring the system. Again, the feasibilty of logging depends. Scanning disks for failures is good, but you must be sure the job HAS run, and what the outcome has been. Even if there were no errros found: send a message the disks seem to be Ok.

    But it depends. Sometimes, a message on each run isn’t feasable. I have a job run every 15 minutes or so, scanning the system for teh MySQL server, to restart it when it has failed. I DID have an issue with that job, where resubmitting itself silently failed, and when MySQL actually crahed, I had to wait all day before I was able to restart it – days after…It would have been noticed if I had received a message of the failure (Ok, I got it – in the logfile that I should have checked…) This has been altered – and the logfile is now in the operator webspace, like all other logs and utilities.