A new VM. Twice

This new VM – aside the first one – is slow as well, doesn’t react on clicks of the mouse, and keyboard input isn’t speedy either. Stopped it, choose another display (VMSVGA in stead of VBoxSVGA), but that made no difference. It might have been something with the VBox extentions, so installed that; But that ruined it all: Screen was completely ruined: no pixels but stripes, nothing could be determined what was on screen; nor was there any reaction on mouse or keyboard, so the only thing I could do was stop it – and removed the machine and create it again from scratch – with two changes: I switched off I/O APIC, because it would show down the VM:, and enabled EFI – because the CD-images had an EFI folder so it is likely it could be used.
This indeed seems to be a good move – so far. The system is way more responsive, especially on IO: installation used about 30 minutes to complete, and no trouble on response, mouse works like I expect.
So saved it, run the updates.

More attemps building SciPy on Linux

It hasn’t done anything. Only showing after stopping the VM because it didn’t response to any command – keyboard nor mouse. Examinaing the logfile shows it (as I expected) wants some answers:
Tasks:
i - Show python/platform/machine information
ie - Show environment information
c - Show C compilers information
c - Set C compiler (current:None)
f - Show Fortran compilers information
f
- Set Fortran compiler (current:None)
e - Edit proposed sys.argv[1:].

Task aliases:
0 - Configure
1 - Build
2 - Install
2 - Install with prefix.
3 - Inplace build
4 - Source distribution
5 - Binary distribution

and error file had some as well:
/usr/lib64/python2.7/distutils/dist.py:267: UserWarning: Unknown distribution option: 'python_requires'
warnings.warn(msg)
Running from scipy source directory.
/usr/lib64/python2.7/site-packages/numpy/distutils/system_info.py:491: UserWarning:
Atlas (http://math-atlas.sourceforge.net/) libraries not found.
Directories to search for the libraries can be specified in the
numpy/distutils/site.cfg file (section [atlas]) or by setting
the ATLAS environment variable.
self.calc_info()
/usr/lib64/python2.7/site-packages/numpy/distutils/system_info.py:491: UserWarning:
Lapack (http://www.netlib.org/lapack/) libraries not found.
Directories to search for the libraries can be specified in the
numpy/distutils/site.cfg file (section [lapack]) or by setting
the LAPACK environment variable.
self.calc_info()
/usr/lib64/python2.7/site-packages/numpy/distutils/system_info.py:491: UserWarning:
Lapack (http://www.netlib.org/lapack/) sources not found.
Directories to search for the sources can be specified in the
numpy/distutils/site.cfg file (section [lapack_src]) or by setting
the LAPACK_SRC environment variable.
self.calc_info()
Traceback (most recent call last):
File "setup.py", line 492, in
setup_package()
File "setup.py", line 488, in setup_package
setup(**metadata)
File "/usr/lib64/python2.7/site-packages/numpy/distutils/core.py", line 152, in setup
config = configuration()
File "setup.py", line 395, in configuration
raise NotFoundError(msg)
numpy.distutils.system_info.NotFoundError: No lapack/blas resources found.

It is obvious that LAPACK BLAS and ATLAS were not found. So I have to set this up -as I found in May:
BLAS=None LAPACK=None ATLAS=None python setup.py build
which I tried, but that gave the same output as in the error file. Perhaps I made a mistake? If so, just run it and get the stuff in files:
BLAS=None; LAPACK=None; ATLAS=None; LAPACK_SRC=None; python setup.py build >scipybld3.log 2> scipy3.err
but even that makes no difference. So would I need to configure:
BLAS=None; LAPACK=None; ATLAS=None; LAPACK_SRC=None; python setup.py configure
but here I get a different error:
setup.py:377: UserWarning: Unrecognized setuptools command, proceeding with generating Cython sources and expanding templates
warnings.warn("Unrecognized setuptools command, proceeding with "
/usr/lib64/python2.7/distutils/dist.py:267: UserWarning: Unknown distribution option: 'python_requires'
warnings.warn(msg)
usage: setup.py [global_opts] cmd1 [cmd1_opts] [cmd2 [cmd2_opts] ...]
or: setup.py --help [cmd1 cmd2 ...]
or: setup.py --help-commands
or: setup.py cmd --help

error: invalid command 'configure'
Duh. I could have known because in an earlier try – where I entered task “0”, the output shows:
Proposed sys.argv = ['setup.py', 'config']
so now I know what to pass – and sending out to files again:
BLAS=None; LAPACK=None; ATLAS=None; LAPACK_SRC=None; python setup.py config >scipycfg2.log 2> scipycfg2.err
there are no errors, and Log seems OK.

So done the build once more, but the result is the same as before. As the errro files indicates, I may have to adjust a file (numpy/distutils/site.cfg). But V is so slow and non-responsive at that moment, it cannot be used. So setup a new one, specifying not “Another Linux’ but ‘Redhat’ since CentOS is the ‘community’ version of this. But again, this takes a lot of time so I let it run overnight.

Building SciPy on CentOS

Today I got the administrative papers. So it’s official – kind of. Need to set up things for this, and contact the tax office…..
Well, apart from that: wanted to build scipy on Linux, but starting the VM failed. To be expected, I moved the disk image to another location, once that was done the VM started – but is very, very slow in acting on mouse movements and clicks….
Installed scipy-1.2.1 as to be used on VMS, by simply extracting the files into a directory. But the documentation won’t tell me on how to build it all, only how to get info on the fortran compiler – with a number of warnings:

$ cd scipy-1.2.1
$ python setup.py config_fc --help-fcompiler
setup.py:377: UserWarning: Unrecognized setuptools command, proceeding with generating Cython sources and expanding templates
warnings.warn("Unrecognized setuptools command, proceeding with "
/usr/lib64/python2.7/distutils/dist.py:267: UserWarning: Unknown distribution option: 'python_requires'
warnings.warn(msg)
Gnu95FCompiler instance properties:
archiver = ['/usr/bin/gfortran', '-cr']
compile_switch = '-c'
compiler_f77 = ['/usr/bin/gfortran', '-Wall', '-ffixed-form', '-fno-
second-underscore', '-fPIC', '-O3', '-funroll-loops']
compiler_f90 = ['/usr/bin/gfortran', '-Wall', '-fno-second-underscore',
'-fPIC', '-O3', '-funroll-loops']
compiler_fix = ['/usr/bin/gfortran', '-Wall', '-ffixed-form', '-fno-
second-underscore', '-Wall', '-fno-second-underscore', '-
fPIC', '-O3', '-funroll-loops']
libraries = ['gfortran']
library_dirs = []
linker_exe = ['/usr/bin/gfortran', '-Wall', '-Wall']
linker_so = ['/usr/bin/gfortran', '-Wall', '-Wall', '-shared']
object_switch = '-o '
ranlib = ['/usr/bin/gfortran']
version = LooseVersion ('4.8.5')
version_cmd = ['/usr/bin/gfortran', '--version']
Fortran compilers found:
--fcompiler=gnu95 GNU Fortran 95 compiler (4.8.5)
Compilers available for this platform, but not found:
--fcompiler=absoft Absoft Corp Fortran Compiler
--fcompiler=compaq Compaq Fortran Compiler
--fcompiler=g95 G95 Fortran Compiler
--fcompiler=gnu GNU Fortran 77 compiler
--fcompiler=intel Intel Fortran Compiler for 32-bit apps
--fcompiler=intele Intel Fortran Compiler for Itanium apps
--fcompiler=intelem Intel Fortran Compiler for 64-bit apps
--fcompiler=lahey Lahey/Fujitsu Fortran 95 Compiler
--fcompiler=nag NAGWare Fortran 95 Compiler
--fcompiler=pathf95 PathScale Fortran Compiler
--fcompiler=pg Portland Group Fortran Compiler
--fcompiler=vast Pacific-Sierra Research Fortran 90 Compiler
Compilers not available on this platform:
--fcompiler=hpux HP Fortran 90 Compiler
--fcompiler=ibm IBM XL Fortran Compiler
--fcompiler=intelev Intel Visual Fortran Compiler for Itanium apps
--fcompiler=intelv Intel Visual Fortran Compiler for 32-bit apps
--fcompiler=intelvem Intel Visual Fortran Compiler for 64-bit apps
--fcompiler=mips MIPSpro Fortran Compiler
--fcompiler=none Fake Fortran compiler
--fcompiler=sun Sun or Forte Fortran 95 Compiler
For compiler details, run 'config_fc --verbose' setup command.
$
Dus bouwen met gnu95, op dezelfde manier als Numpy: uitvoer naar bestanden
(terminal sessie houdt muis vast? Of is dat de VM zelf?
$ python setup.py config_fc --fcompiler=gnu95 install > scipy.log 2> scypi.err
Die is snel klaar:
$ cat scipy.log

Note: if you need reliable uninstall behavior, then install
with pip instead of using `setup.py install`:

- `pip install .` (from a git repo or downloaded source
release)
- `pip install scipy` (last SciPy release on PyPI)

lapack_opt_info:
lapack_mkl_info:
mkl_info:
libraries mkl,vml,guide not found in ['/usr/lib64']
NOT AVAILABLE

NOT AVAILABLE

system_info:
NOT AVAILABLE

atlas_threads_info:
Setting PTATLAS=ATLAS
libraries tatlas not found in /usr/lib64/atlas
libraries lapack_atlas not found in /usr/lib64/atlas
numpy.distutils.system_info.atlas_threads_info
NOT AVAILABLE

atlas_info:
libraries tatlas not found in /usr/lib64/atlas
libraries lapack_atlas not found in /usr/lib64/atlas
numpy.distutils.system_info.atlas_info
NOT AVAILABLE

lapack_info:
libraries lapack not found in ['/usr/lib64']
NOT AVAILABLE

lapack_src_info:
NOT AVAILABLE

NOT AVAILABLE

$

That’s all. But now I know I have gnu95 Fortran compiler on this box; no idea what about the other messages…
To get info on build, the documentation refers to the site (scipy.org) but nothing on building the basic way. So back to the files/
Since numpy is from the same organization, it is to be expected that a very basic installation is:

$ python setup.py > scipybld.log 2> scipybld.err

Started that at about 23:00 – and it may run for hours, so keep it running overnight.

Numpy built on Linux – got my commands. And Brett’s DCL file

Build Numpy on the Linux machine (just numpy, leave the rest for now):

python setup.py build -j 1 install --prefix $HOME/.local >build.log 2>build.err

and copied the resulting outputfiles to workstation to examine.
Brett has corrected the login on CTRL-C so I can login again; rolled back the backup file he placed there, and copied that as well to find out what he created. His environment is Python 3.5 – not 2.7 – but that should not be a problem – some changes will be needed anyway. Checked this against what my own build on Linux made of it: pretty much the same (this is Python 2.5 – but the build procedure did not complain)

Setting up Linux environment as a testbed for compilation commands

To build the Numpy C files, I need to get some documentation on C and C++ – but this is missing on the VSI site. Asked Brett, got that – plus what he had done on this in his spare time: He installed the bunch on a Windows PC running Cygwin, started a build (in C only) and trapped the output in a file; so he could transfer that into a plain, straightforward DCL procedure that compiled it all, placed modules in a library, and created the shared image. He passed it all in a backup file on the CTRL-C cluster – but this has been updated to VMS OpenVMS, and my login failed – because I was denied access to two files.. Asked him to adjust….
In the mean time, this is the way to go: Build in an environment that is known to work (some way or another), pick up the output and translate that into DCL – straight-forward, without any intelligence to be gin with.
So I set up a virtual machine running CentOS 7, installed all updates which caused a bunch of issues; but finally I had a more or less stable Linux environment where I installed the packages (OpenBLAS, NumPy en SciPy) and on VMS, I installed latest versions of MMK and Make from the VMS Freeware CD’s (that I have online) – may not be the latest but for now this will do.

Had a mail from Guido: proceed with this set up: Base and additions apart, add another layer for my own changes – that means that the adapted MMS files will reside in yet another layer.