Jesus Loves GRASS

WMS/WFS getfeature on hover, problem to display the GML (mapserver)

2009-09-29T13:05:00.005+01:00

I had a hard time programming something similar to the "WFS GetFeature Example" shown in the OpenLayers examples, the example uses calls to a WFS/WMS services that have the same name and data origin.

The WFS/WMS service call is done with the OpenLayers.Protocol.WFS.fromWMSLayer as follows (the layerWMS is a OpenLayrs.Layer.WMS object)

control = new OpenLayers.Control.GetFeature({
protocol: OpenLayers.Protocol.WFS.fromWMSLayer(layerWMS),
hover: true
});

Using firebug it is possible to see the requests going back-and-forward when hovering the polygons, but in the end the polygons aren't select, because trigger-events aren't called.

The problem is cause by the namespace used in the WFS of mapserver (since the example given in the web site uses geoserver, everything is ok), the solution is to indicate the namespace and featureTypeName.

In my mapserver response I have something like this:
<gml:featureMember>
<ms:wdpa gml:id="wdpa.67886">

Therefore control has to be changed as follows:

control = new OpenLayers.Control.GetFeature({
protocol: OpenLayers.Protocol.WFS.fromWMSLayer(layerWMS,{featureType:"wdpa",featurePrefix:"ms"}),
hover: true
});

It seems that the same has to be done for the standard WFS layer when using mapserver.

white space fore php tag

2009-07-30T15:25:00.004+01:00

While a blank line before the <?php start tag may look innocent, when processed by PHP, it will turn into an echo statement printing out a blank line.

If working with web service wrapper scripts from mapserver this could be a disaster resulting in some strange behaviours like:

1) After $request->loadparams(); the script will send you an empty file as a reply
2) If the $request has uses setparameters, all the outputs will be dumped to the browser window, for example a WMS GetMap request will be a binary string in the browser window.
3) The ms_iogetStdoutBufferBytes(); will return nothing if you want to save the image locally.
4) Output with Warning: Cannot modify header information - headers already sent by

These symptoms are caused by the existence of an open buffer cause by the blank line, at the begining of the php programming.

Problem with Tomcat 5.5. HTTP Status 404 - The requested resource (...) is not available. (mod_jk)

2009-04-29T08:11:00.004+01:00

Dealing with tomcat errors can be a pain in the neck.

For example a situation like this:

1) Considering that tomcat is running and apache's mod_jk is being used to connect to it.
a. Tomcat works fine when using http://mysite.com:8180/mypage
b. http://mysite.com/mypage generates a HTTP error 404 with resource not found.

2) Probably there is a miss match between the httpd.conf file and the server.xml configuration file
- ServerName, server IP or ports may be different in the 2 configuration files
- If you recompiled Apache, then you need to recompile mod_jk

If you are a newbie in tomcat (like me) this error may take some time to figure it out, since the error logs of apache are clean and the tomcat error logs don't help much

Error on CGI script

2009-01-21T08:47:00.002Z

An error like this:
(8)Exec format error: exec of '/usr/local/apache2/cgi-bin/CGI2WPS.py' failed
[Wed Jan 21 09:38:41 2009] [error] [client 127.0.0.1] Premature end of script headers: CGI2WPS.py

Means that the execution of the script failed, probably the server has some problems on getting the python interpreter.

Adding the python location to the script will solve it:

#!/usr/bin/python
#-*- coding: utf-8 -*-

gdalinfo 1.4.2 versus gdalinfo 1.5.2

2008-08-29T10:02:00.003+01:00

While processing some gdalinfo string-to-array in PHP I noticed that the output of gdalinfo is different from 1.4 to 1.5 and therefore my code broke in a new machine.
Gdal 1.4.2:
Driver: GTiff/GeoTIFF
Size is 35, 72
Coordinate System is `'
Origin = (-72000.000000000000000,

643700.000000000000000)
Pixel Size = (9422.857142857143117,-9565.277777777777374)
Corner Coordinates:
Upper Left ( -72000.000, 643700.000)
Lower Left ( -72000.000, -45000.000)
Upper Right ( 257800.000, 643700.000)
Lower Right ( 257800.000, -45000.000)
Center ( 92900.000, 299350.000)
Band 1 Block=35x58 Type=Float32, ColorInterp=Gray
Band 2 Block=35x58 Type=Float32, ColorInterp=Undefined
Band 3 Block=35x58 Type=Float32, ColorInterp=Undefined

Gdal 1.5.2
Driver: GTiff/GeoTIFF
Files: geotiff_data2087566292
Size is 35, 72
Coordinate System is `'
Origin = (-72000.000000000000000,643700.000000000000000)
Pixel Size = (9422.857142857143117,-9565.277777777777374)
Image Structure Metadata:
INTERLEAVE=BAND
Corner Coordinates:
Upper Left ( -72000.000, 643700.000)
Lower Left ( -72000.000, -45000.000)
Upper Right ( 257800.000, 643700.000)
Lower Right ( 257800.000, -45000.000)
Center ( 92900.000, 299350.000)
Band 1 Block=35x58 Type=Float32, ColorInterp=Gray
Band 2 Block=35x58 Type=Float32, ColorInterp=Undefined
Band 3 Block=35x58 Type=Float32, ColorInterp=Undefined

The new version includes:
Image Structure Metadata:
INTERLEAVE=BAND

and the name of the file that was read
Files: geotiff_data2087566292

Paper for GI-Days 2008 (Munster)

2008-05-19T08:37:00.003+01:00

Just submitted a paper concerning WPS and the AutoMap (Gstat) package. The full paper will be online after the conference, meanwhile here is the abstract:

Web-based geostatistics using WPS

Jorge de Jesus1, Grégoire Dubois2, Paul Hiemstra3
1Institute for Environment and Sustainability / Joint Research Centre of the European Commission
(email removed)
1Institute for Environment and Sustainability / Joint Research Centre of the European Commission
(email removed)
3Department of Physical Geography / University of Utrecht
(email removed)

Abstract.

Mapping data using geostatistics can be a time-consuming process because of the many parameters to define. automap, a geostatistical package written in R, was developed to define automatically a spatial correlation model, a step that is considered to be the biggest obstacle for automating the spatial interpolation process with geostatistical algorithms. The implementation of automap into a Service Oriented Architecture (SOA) would ensure the automatic interpolation routines to be platform independent, easy to integrate into existing applications, and freely available in a network environment. automap is here further proposed as a Web Processing Service (WPS) to allow end-users to benefit from a remote service to be used for the real-time mapping of environmental data in routine situations. It is the purpose of this paper to present the architecture of the service as well as to discuss by means of case studies the limitations of the real time mapping system for emergency response.

The image .... cannot be displayed, because it contains errors.

2008-02-27T10:36:00.004Z

"The image 'http://localhost/ACI/test.php' cannot be displayed, because it contains errors." Was the only thing I was getting when running some examples and source codes for image creation using GD and PHP. I googled it and all the information found concerned bad installation of GD and other libraries.

But when dumping the GD info with var_dump(gd_info()); everything was ok.

In the end I discovered that actually the error is not related to GD but with the header used to send the image, this example works fine:

<?php
header("Content-type: image/png");
$im = imagecreate(210, 20) or die("Cannot Initialize new GD image stream");
$background_color = imagecolorallocate($im, 0, 0, 0);
$text_color = imagecolorallocate($im, 25, 25, 25);
imageline($im,10,10,20,20,$text_color);
imagepng($im);
imagedestroy($im);
?>

But something like this:
<?php
//Something, source or text
?>
<?php
header("Content-type: image/png");
$im = imagecreate(210, 20) or die("Cannot Initialize new GD image stream");
$background_color = imagecolorallocate($im, 0, 0, 0);
$text_color = imagecolorallocate($im, 25, 25, 25);
imageline($im,10,10,20,20,$text_color);
imagepng($im);
imagedestroy($im);
?>

Will only generate the stupid GD error message.

The header() needs to be the first thing in the PHP code as explained in the PHP doc "Remember that header() must be called before any actual output is sent, either by normal HTML tags, blank lines in a file, or from PHP. It is a very common error to read code with include(), or require(), functions, or another file access function, and have spaces or empty lines that are output before header() is called. The same problem exists when using a single PHP/HTML file."

$oMap->setExtent problems and general data gathering

2008-02-26T08:32:00.003Z

I made some PHP programming/MapScript for rendering a GeoTiff created by an interpolation service (WPS) as a PNG. The Geotiff is stored in the server's temporary folder and all the mapserver parameters are set by PHP programing.

The fist part of the posted code gets the image info using GdalInfo and regular expressions to obtain the extent value of the image, I haven't managed to do this properly using the PHP's Exif lib....

The major problem was the SetExtent of the Map object it was always complaining that the values had some problem, the problem was that the parameters were being passed as a string, and they needed to be as double, even if they are passed as float (or int?) the SetExtent will complain

//Some code before

$FileName=/usr/local/apache2/htdocs/tmp/interpolation3454.tiff;
$CmdStr="/usr/local/bin/gdalinfo ".$FileName;
//General string for getting float numbers
$StrPattern='/[-+]?\b[0-9]*\.?[0-9]+\b/';
exec($CmdStr,$ResultArray);

//Upper left point info
$UpperLeftPoint=$ResultArray[6];
preg_match_all($StrPattern,$UpperLeftPoint,$TmpArray);
$MinX=$TmpArray[0][0];
$MaxY=$TmpArray[0][1];

//Lower right point info
$LowerRightPoint=$ResultArray[9];
preg_match_all($StrPattern,$LowerRightPoint,$TmpArray);
$MaxX=$TmpArray[0][0];
$MinY=$TmpArray[0][1];

//Passing to Map object
//double minx, double miny, double maxx, double maxy
$oMap->setExtent((double)$MinY,(double)$MinY,(double)$MaxX,(double)$MaxY);

//Same code after

Basic Image in PHP/Mapscript

2008-02-04T16:00:00.001Z

This script makes a simple web page with a PNG image based on a GeoTiff that contains interpolation values in Float32. The script uses the ms_GetErrorObj() to echo any errors.

The objective of the script is to totally replace any info from the map file, nevertheless it is necessary for a map file to start the Map object.

The map file called test2.map is only:
MAP

END

The rest is defined as follows.....

<?php
/*
* Created on Feb 1, 2008
*/

// Loading Lib (just to be certain)
dl('php_mapscript.so');

//Reset error list (just to be certain)
ms_ResetErrorList();

$map_path="/usr/local/apache2/htdocs/wpstmp/";

// Creation of map object
$oMap = ms_newMapObj($map_path."test2.map");

//Setting the name
$oMap->set("name","map1");

//Setting size and backcolor
$oMap->setSize(800,400);
$oMap->imagecolor->setRGB(125,125,125);

//Setting Extent
//Data gathered using GdalInfo from the GeoTIFF
//double minx, double miny, double maxx, double maxy
$oMap->setExtent(-69000.000,-42000.000,261000.000,633000.000);

//Seting units
$oMap->set("units",MS_METERS);

//Output type
$oMap->selectOutputFormat("PNG24");
$oMap->outputformat->setOption("imagemode",MS_IMAGEMODE_RGB);

//Paths
$oMap->web->set("imagepath","/usr/local/apache2/htdocs/wpstmp/");
$oMap->web->set("imageurl","/wpstmp/");

//Layer Object
$oLayer=ms_newLayerObj($oMap);
$oLayer->set("name","interpolation1");
$oLayer->set("status",MS_ON);
$oLayer->set("type",MS_LAYER_RASTER);
$oLayer->set("data","/usr/local/apache2/htdocs/wpstmp/interpolation610385.tif");

//RASTER processing of band1 of Geottif, automatic color scaling
//http://mapserver.gis.umn.edu/docs/howto/raster_data
$oLayer->setProcessing("SCALE=AUTO");
$oLayer->setProcessing("BANDS=1");

//Dumping image
$oImage=$oMap->draw();
$image_url=$oImage->saveWebImage();

//Error dumping part, not very necessary, only for debug
$error=ms_GetErrorObj();
while($error && $error->code!=MS_NOERR)
{
printf("Error in %s: %s
\n",$error->routine,$error->message);
$error=$error->next();
}

?>
<html>
<head>
</head>
<body>

<img src="http://localhost<?php echo $image_url?>" alt="Map" />

</body>
</html>

Personal Note

2008-01-17T07:45:00.000Z

Personal Note:

The Ben-Gurion University of the Negev has given me the degree of Ph.D, since 03 December 2007 I am Doctor Jesus :)

WPS update

2008-01-17T07:40:00.000Z

It seems that I have been a bit lazy to post anything...

I am currently working on a Web Processing Service (WPS) for automatic interpolations (I like to call it RAISIN), that is integrated on the INTAMAP project. Everything is still a bit "buggy" but going on the write direction. The documentation and server can be found at:

http://remwps.jrc.it

and the WPS supporting script:

http://remwps.jrc.it/wps.py

I hope to post some tiny/usefull scripts in the blog any time soon.

By the Love of G-d where are my POST variables !!!!

2007-10-05T13:11:00.000+01:00

I am working in a WPS web service, using mod_python and PyWPS, and all the time I was using GET as HTTP request.

The WPS needs for some requests to be in the form of XML file so there is a need to implement POST to send such requests, for start I decided to try using POST to fetch some Key-Value-Pair.

I used the util class of mod_python as usual to deal with the request (as indicated on the mod_python tutorial:

form=util.FieldStorage(req)

But the form list was empty, tried so many things and so many possibilities like

request_made=req.read()

The only thing I was getting was a null reply, everything empty !!!

I found a post in the mod_python mailing-list with the same problem but no answer....and nothing on the specifications of mod_python.

Finally I discovered the problem !!!! POST was always empty because of PythonHandler mod_python.publisher and by dropping it and using my script file (wps2.py) as handler (PythonHandler wps2)the problem got solved. So the conclusion is: DON'T USE Publisher handler + POST together.

Also there is a huge lack of info on this POST "problem"!!!! This should be referred in the documentation !!!!

Junk after XML (0 at the end of reply)

2007-09-20T16:00:00.001+01:00

I am finally back to programming and I am currently working in providing Web Processing Service.

I started to use PyWPS implementation for it (it is very well done, complements to Jachym Cepicky) but PyWPS runs as a CGI script and not from inside mod_python, the possibility of running it using the cgihandler from mod_python is a limited option.....

So I am making a new version of PyWPS crafted to my needs, I managed to get a Request=GetCapabilities working, but I got stuck with this error:

XML Parsing Error: junk after document element
Location: http://localhost/test/wps/wps2.py/main?version=0.4.0&Request=GetCapabilities
Line Number 104, Column 1:0
^
0

Well it seems there is an "extra" after the XML document, the problem is related to the req object of mod_python and how the HTTP is processed.

This 0 indicates that there is no more HTTP chunks (more or less EOF).

To disable chunked encoding, it is necessary to specify content length:

req.send_http_header()
req.content_type = 'text/xml'
req.set_content_length(len(getCapabilities.document.toprettyxml()))
req.write(getCapabilities.document.toprettyxml())
return apache.OK

The getCapabilities.document.toprettyxml() creates my XML document that then is checked for lenght, after it will be flushed with req.write

putObject object type coercing

2007-08-03T10:26:00.000+01:00

Like it says in the API of RSOAP sometimes it's SOAP has problem converting object and it may need some help:

In the python shell:
>>array_numbers=RSession.call("rnorm",10)

This array_number will be an instance object.

To put this list python object in the R
>>RSession.putObject("x",list(array_numbers))

if there weren't the conversion to list from instance the R server would reply
'None'

PS: Better to use the wrapper function shown in the article RSOAP in Python Zine to deal with object transformation

Creation of a Semi-Variogram using RSOAP and Python

2007-08-02T17:45:00.000+01:00

This is basically what is in Py Zine about RSOAP but with some differences....

Starting in the bash shell by activating the RSOAPManager : ./RSOAPManager --debug

########################################################
STARTING R SOAP Manager on localhost:9081
########################################################
Manager Port: 9081
R Server Port range: [9082 to 65536]

Setting up R Process Object...
../lib/RProcess.py:70: RuntimeWarning: tempnam is a potential security risk to your program
self._tmpdir = os.tempnam(self.path,"RSOAP") + "/"
RHOME= /usr/local/lib/R
RVERSION= 2.5.1
RVER= 2051
RUSER= /root
Loading Rpy version 2051 .. Done.
Creating the R object 'r' .. Done
Setting up SOAP Listener

########################################################
Listening for SOAP requests at http://localhost:9081
########################################################

Waiting for request...

jumping to the python shell:

>>from SOAPpy import *

Starting a new SOAP connection:
>>RSOAPServer=SOAProxy(“http://localhost:9081”) # as indicated by RSOAPManager

To check if the connection is ok
>>RSOAPServer.echo(“something”) #the server will reply to what ever we write
'something'

Requesting a new R session and port information:
>>RSessionURL=RSOAPServer.newServer() # the reply should be http://localhost:9083 if the reply is 'None' it didnt opened the session

It is also possible to keep track of the ports using the nmap in the bash shell:
>nmap -v -p9080-9090 localhost

The making the actual connection to the R session
>>RSession=SOAPProxy(RSessionURL)

checking:
>>RSession.echo(“test”) # for killing session : RSession.quit()

3 major ways to deal with the R session: function call, eval, script. The first one (call) is used to call functions and pass parameters, eval is used to send lines of commands (ex: names(ssa)<-c(“x”,”y”,”z”)) and script to process complete scripts.

For the semivariogram creation it will be used the sgeostat pack. The first thing is to load the library

>>RSession.call(“library”,”sgeostat”) #fist it is called the functions and then the parameters to pass

Using sgeostat it is necessary the following steps to create a semivariogram
-Upload data
-Create point object
-Create pair object
-Estimate the semi-variogram

For loading the data set and creating the data object (ssa):
>>RSession.eval("ssa<-read.csv(file='/home/jesus/doutoramento/db/SSA.csv',header=FALSE,sep=';')")

The eval will pass the command R.

to see the new object:
>>RSession.call(“ls”)
'ssa'

For the other sequential commands it is better to use the script option. Normally for this method it is assumed the \n as the escape char for separation, it also allows for prompts when the echo option is activated.

script_text="names(ssa)<-c('x','y','z') \n ssa.point<-point(ssa) \n ssa.pair<-pair(ssa.point) \n ssa.estvar<-est.variogram(ssa.point,ssa.pair,'z')"

Sending the script:
>>Rsession.script(script_text)

Checking:
>>RSession.call("ls")
['resultText', 'ssa', 'ssa.estvar', 'ssa.pair', 'ssa.point']

Normally to output the result I use png() to save the result as a png file, the problem is that X11() device is deactivated (it took me sometime to discover....), to see what device is being used:

>>RSession.call(“getOption”,”device”)
'postscript'

Another possibility is to use the PDF output
>>> RSession.eval("options(device='pdf')")

Using the PDF the result will be dumped using a script that opens the PDF device, plots, closes/saves
>>script_text2="pdf(file='/home/jesus/tmp/sv.pdf') \n plot(ssa.estvar) \n dev.off()"
>> RSession.script(script_text2)

The result should be the SV PDF image with the semivariogram :)

RSOAP server

2007-08-02T17:33:00.000+01:00

R is excellent for advance calculations like geostatistics or data set processing. In modern days with web 2.0 were everything is done in the web and not on the desktop it would be practical to have R as a server to process requests.

R can integrate in common internet languages like Python, Perl and PHP (this integration is still a bit weak....). These languages simple use R as an extension/module of their own features, another option is to access R as a web service, allowing for R to be a server replying to the requests of clients (what ever the client might be)

SOAP is a protocol for exchanging XML messages over networks, these messages can contain data, execution calls etc., using the RSOAP module (that was developed using RPy) it allows for R to work as a server replying to requests made using this protocol.

The RSOAP 1.1.4 depends on R>=1.4.1 ; Rpy >=0.3.0 ; PyXML >=0.7 ; Python >= 2.1 ; SOAPpy >= 0.9.8 and R-library session >= 1.0

All the necessary tools are simple to install and don't give major headaches.... (python setup.py install tango dance....)

The RSOAP runs as total independent process listening to a specific port. In the script directory or RSOAP (in my case: /usr/local/src/RSOAP-1.1.4/scripts) there is the RSOAPManager script that activates the RSOAP server. As specifications it is possible to define the port to be used as the SOAP server and to the lower and upper limit of the R session ports, and of course the always useful debug options

Basically RSOAP starts a SOAP server at port 9081, this port listens for requests of R sessions. After a request for a session is made, the server opens a port (starting with 9082+1 till 65535 ) for R command listening, this allows for 1 SOAP server to process different sessions from different clients at different ports.

Using the –debug option the .config.dumpSOAPout=1 and .config.dumpSOAPin=1 (options of the SOAP server object) will be activated dumping all the SOAP requests to the shell for debugging, this is very useful....

SOAPpy small bug

2007-07-31T15:36:00.000+01:00

Installing SOAPpy 0.12.0 on python 2.5 I got the following error:

Traceback (most recent call last):
File "setup.py", line 8, in
from SOAPpy.version import __version__
File "/usr/local/src/SOAPpy-0.12.0/SOAPpy/__init__.py", line 5, in
from Client import *
File "/usr/local/src/SOAPpy-0.12.0/SOAPpy/Client.py", line 46
from __future__ import nested_scopes
SyntaxError: from __future__ imports must occur at the beginning of the file

The solution is to open file Client.py and copy/past the "from __future__ imports" line to be the first import.

The same error will appear in several files and also during installation, but the solution is always the same

Easier way to check python

2007-07-26T08:24:00.000+01:00

Like we say in Portuguese "Who knows, know it" here is an easier way to get the mod_python info (posted by Grahan Dumpleton in the comments)

It is easier to go:

<Location /mpinfo>
SetHandler mod_python
PythonHandler mod_python.testhandler
</Location>

and then use '/mpinfo' as URL.

mod_python info server request

2007-07-25T13:53:00.000+01:00

This is how information of the server and options can be made with mod_python 3.3.1 (like to do phpinfo() in PHP)

1. Make a /test directory in the htdocs tree

2. Create a directory structure in the httpd.conf as follow:

<Directory "/usr/local/apache2/htdocs/test">
Options Indexes FollowSymLinks
AllowOverride All
AddHandler mod_python .py
PythonHandler mod_python.testhandler
PythonDebug On
Order allow,deny
Allow from all
</Directory>

3. Restart Apache

4. Point to some hypothetical python script: http://localhost/test/ImaginaryScript.py

And the information of mod_python and apache should appear!!!!

IE header problem in Ka Map print_map.php

2007-06-12T16:35:00.000+01:00

I was thinking that everything was ok with the printing system, but yesterday I was with my supervisors and he used Internet Explorer, when he tried to download and image file with the screen content of Ka-maps, the IE said:
"Internet Explorer cannot download...cale12000000&output_type=JPEG"

"Internet Explorer was not able to open this Internet site"

Since I saw the & and some GET like structure I thought it was some PHP error....

The problem is in the header creation of the print_map.php, which works fine for Firefox but it has some problems with IE.

I have searched on the PHP tutorial help and read several things and so far the best header is something like this:

$mm_type="image".strtolower($_REQUEST['output_type']);
$filename_ext=$oMap->name . ".".strtolower($_REQUEST['output_type']);
header("Cache-Control: public, must-revalidate\n");
header("Expires: Mon, 26 Jul 1997 05:00:00 GMT\n");
header('Last-Modified: ' . gmdate( 'D, d M Y H:i:s' ) . ' GMT'.'\n');
header("Cache-Control: no-store, no-cache, must-revalidate\n");
header("Cache-Control: post-check=0, pre-check=0\n", false );
header("Pragma: Internet Explorer is shitty\n"); //Otherwise it will give error
header("Content-Type: " . $mm_type."\n");
header("Content-Length: " .(string)(filesize($szImg))."\n");
header('Content-Disposition: attachment; filename='.$filename_ext."\n");
header("Content-Transfer-Encoding: binary\n");
fpassthru($h);
fclose($h);

-The headers for what I read are prepared for non-cache in HTTP 1.0 and 1.1
-Without the Pragma: (something), IE will give the same error, I haven't found why, but is what people do
- The \n is seems to be necessary for IE7 that needs a more tight header (but I haven't checked if it's true)

XML parse problem with getElementsByTagName()

2007-06-10T14:51:00.000+01:00

Today a college checked the WebGIS and she noticed the WCS request was not working. With me everything was fine but not with her, the difference is that I use Firefox while she is on IE6

So I have discovered that there is a problem parsing the XML of the describeCoverage request of the WCS.

I was using something like this to get the pixel resolution from the offsetVector information of the XML reply

var RESnode=xmldoc.getElementsByTagName("offsetVector");
var sRESX=RESnode[0].firstChild.nodeValue;
var RESX=sRESX.split(" "); //array with resx in this case the first value is the important one
var sRESY=RESnode[1].firstChild.nodeValue;
var RESY=sRESY.split(" ");

This gets the X,Y resolution of the image using FireFox

The problem is that tag offsetVector is in reality gml:offsetVector, as the specification of the WCS is to include the type of XML structure, well FireFox doesn't care about anything before the :

I have found a simple solution in this blog:
http://learningremix.net/w2007integ/jeff/2007/03/re_problem_related_to_parsing_xml.shtml

So in case the XML reply is empty I am using another tag name:

var RESnode=xmldoc.getElementsByTagName("offsetVector");
if (RESnode.length == 0) {
var RESnode = xmldoc.getElementsByTagName("gml:offsetVector");
}
var sRESX=RESnode[0].firstChild.nodeValue;
var RESX=sRESX.split(" "); //array with resx in this case the first value is the important one
var sRESY=RESnode[1].firstChild.nodeValue;
var RESY=sRESY.split(" ");

PROBLEM SOLVED !!!

Mapserver Versus Google Earth

2007-06-04T14:12:00.000+01:00

Thinking of using google earth as a WMS layer in your mapserver project ??! Then think again !!!

Gregor Mosheh
Reply-To: Gregor Mosheh
Date: Thu, 1 Mar 2007 10:17:58 -0700
Content-Type: text/plain

There's no way to use Google Maps in Mapserver, due to Google's terms of
use. Some folks wrote a WMS gateway to Google Maps, for example, and got a
cease-and-desist for it.

There are other options, though, if you're willing/able to serve up your
Mapserver output via WMS. A) You could use Mapserver's WMS output in a
Google Map, because Google doesn't have a problem with that. B) Look for a
framework that supports both WMS and Google Maps and use that to import
both your WMS output and Google's basemap; I know that OpenLayers had
Google Maps support on their drawing board, but I don't know the status of
it.

Source: Mapserver Mailing List

google maps

2007-05-25T14:45:00.000+01:00

I had to make a google map with the locations of hotel for the participants of the AEMP2007 congress. The API of google maps is very simple:
1) Creation of the map including center point and zoom used
2) Include controls
3) Creation of the Icons to be used
4) Adding a point to the map and making a marker (clicking the point it will display HTML)
5) Div section with the map on the webpage

There is a nice webpage with google maps icons:
http://www.econym.demon.co.uk/googlemaps/geicons.htm

1) Creation of the map:
function load() {
if (GBrowserIsCompatible()) {
var map = new GMap2(document.getElementById("map"));
map.setCenter(new GLatLng(37.03339,-7.95112), 13);
map.setMapType(G_HYBRID_TYPE);

2) Include controls
map.addControl(new GSmallMapControl());
map.addControl(new GMapTypeControl(true));

3) Creation of Hotel icon

var iconHotel = new GIcon();
iconHotel.image = "http://maps.google.com/mapfiles/kml/pal2/icon20.png";
iconHotel.shadow = "http://maps.google.com/mapfiles/kml/pal2/icon20s.png";
iconHotel.iconSize = new GSize(32, 32);
iconHotel.shadowSize = new GSize(37, 34);
iconHotel.iconAnchor = new GPoint(9, 34);
iconHotel.infoWindowAnchor = new GPoint(9, 2);
iconHotel.infoShadowAnchor = new GPoint(18, 25);

4) Adding Point and marker
var pointIbis = new GLatLng(37.03339,-7.95112);
var markerIbis=new GMarker(pointIbis,iconHotel)
map.addOverlay(markerIbis);
GEvent.addListener(markerIbis, "click", function() {
markerIbis.openInfoWindowHtml("<b>Hotel Ibis</b>");
}); //don't forget this

and closing the load function and if statement:
}
}

- The function load() is called inside the body tag with the onload event
<body onload="load()">

- The DIV in the page couldn't be simpler
<div id="map" style="width: 600px; height: 400px"></div>

GML information in array form

2007-05-18T21:02:00.000+01:00

The GML information is structured like this in the $key (dump programmin in the post below)

Keys array

Array
(
[SCHEMA] => Array
(
[0] => 0
[1] => 2
[2] => 4
[3] => 50
[4] => 51
)

[IMPORT] => Array
(
[0] => 1
)

[ELEMENT] => Array
(
[0] => 3
[1] => 9
[2] => 11
[3] => 13
[4] => 15
[5] => 17
[6] => 19
[7] => 21
[8] => 23
[9] => 25
[10] => 27
[11] => 29
[12] => 31
[13] => 33
[14] => 35
[15] => 37
[16] => 39
[17] => 41
)

[COMPLEXTYPE] => Array
........................ (more) .............

The number indicated by the key and the nested arrays is the location of of the specific tags inside the values array, for example the data from the first ELEMENT Tag will be in position 3 of the $Values array ([ELEMENT] => Array([0] => 3....)
Values array

Array
(
:
:
:
[3] => Array
(
[tag] => ELEMENT
[type] => complete
[level] => 2
[attributes] => Array
(
[NAME] => major_cities
[TYPE] => ms:major_citiesType
[SUBSTITUTIONGROUP] => gml:_Feature
)

)
:
:

An example to fetch the NAME data from the first ELEMENT would be:
$values[$keys["ELEMENT"][0]]["attributes"]["NAME"]

or using a more sofisticate approach with the point functions of PHP:
$values[reset($keys["ELEMENT"])]["attributes"]["NAME"]

This is not as sofisticates as the XML-DOM system but it works fine.

WFS reply to DescribeFeatureType

2007-05-18T20:45:00.000+01:00

This is the GML version 2 that the DescribeFeatureType will produce from a layer containing the most important cities of the study area:

<schema targetNamespace="http://mapserver.gis.umn.edu/mapserver" elementFormDefault="qualified" version="0.1">
<import namespace="http://www.opengis.net/gml" schemaLocation="http://schemas.opengis.net/gml/2.1.2/feature.xsd"/>
<element name="major_cities" type="ms:major_citiesType" substitutionGroup="gml:_Feature"/>
<complexType name="major_citiesType">
<complexContent>
<extension base="gml:AbstractFeatureType">
<sequence>
<element name="msGeometry" type="gml:GeometryPropertyType" minOccurs="0" maxOccurs="1"/>
<element name="cat" type="string"/>
<element name="AREA" type="string"/>
<element name="PERIMETER" type="string"/>
<element name="CITIES_" type="string"/>
<element name="CITIES_ID" type="string"/>
<element name="CITY_NAME" type="string"/>
<element name="GMI_ADMIN" type="string"/>
<element name="ADMIN_NAME" type="string"/>
<element name="FIPS_CNTRY" type="string"/>
<element name="CNTRY_NAME" type="string"/>
<element name="STATUS" type="string"/>
<element name="POP_RANK" type="string"/>
<element name="POP_CLASS" type="string"/>
<element name="PORT_ID" type="string"/>
<element name="AREA_1" type="string"/>
<element name="LEN" type="string"/>
</sequence>
</extension>
</complexContent>
</complexType>
</schema>

WFS XML verbose to Array structure

2007-05-18T20:32:00.000+01:00

I was expecting for the WFS reply to be more "friendly" meaning the Mapserver dump in GML to be easily openned by other programs like ArcGIS. But is not the case....since my supervisor wants the results to be in CSV I have to send sometime on it.

Meanwhile I have a nice way of concerting the mapserver WFS reply to an array structure using PHP XML Parser (I should have used the XML-DOM parser but it was a bit to advanced..., also I couldn't find any nice class to deal with it in the website php classes)

(programming requesting the WFS layer)

$buffer = ms_iogetstdoutbufferstring();
ms_ioresethandlers();

//creation of the values and keys containing the DescribeFeatureType or GetFeature
$xp = xml_parser_create();
xml_parse_into_struct($xp,$buffer,$values_meta,$keys_meta);
xml_parser_free($xp);

//dumping all the array structure for checking
echo "Keys array_meta<br><br><pre>";
print_r($keys_data);
echo "</pre><br><br>Values array_meta<br><br><pre>";
print_r($values_data);
echo "</pre>";

The example is from the book "Beginning PHP5" and it was the simplest way I found to pass everything to arrays

I can't belive it I got the job !!!!

2007-05-13T11:00:00.000+01:00

I was a couple of days in the Alentejo region working on a research field and had to take care of some temperature logger and collect soil samples for a project. I discovered that the field was infested with ticks, after I got home by the end of the week I discovered 3 ticks on my body, it was a bit painful to take the bitches out with alcohol.

But the good news was that I got the job at the JRC in Ispra, I just have to pass a medical exam and to send some documentation and if everything goes according to plan I am starting at the 16th July.

This means that I have to finish the WFS system in the CALTER project As Soon As Possible!!!

PHP popup

2007-04-20T17:48:00.000+01:00

This is the PHP that will make the popup, it is more or less prepared for when the WFS is installed:

<?php
/* This PHP will generate the HTML code for the final submission of the request
The Javascript calling string is :
url_str='popup.php?service='+OWS_service+'&request=getCoverage&COVERAGE='+OWS_name+'&CRS='+CRS+'&BBOX='+BBOX.join(",")+'&RESX='+RESX[0]+'&RESY='+(-1*parseFloat(RESY[1]))+'&FORMAT='+FORMAT; */

?>

<body>
<table width="350px" border="0">
<tbody>
<tr bgcolor="#CFCFCF" >
<td><?php echo(strtoupper($_GET["service"])); ?> name:</td>

<?php if (strnatcasecmp ($_GET["service"],'wcs') == 0){
$OWS_name=$_GET["COVERAGE"];
}
?>
<td align="center" ><?php echo($OWS_name); ?></td>
</tr>
<tr>
<td>CRS (projection):</td>
<td align="center"><?php echo($_GET["CRS"]); ?></td>
</tr>
<tr bgcolor="#CFCFCF">

<?php
$BBOX=explode(",",$_GET["BBOX"]); ?>
<td>BBOX minX:</td>
<td align="center"><?php echo($BBOX[0]); echo(" º N"); ?></td>
</tr>
<tr>
<td>BBOX minY:</td>
<td align="center"><?php echo($BBOX[1]); echo(" º N"); ?></td>
</tr>
<tr bgcolor="#CFCFCF">
<td>BBOX maxX:</td>
<td align="center"><?php echo($BBOX[2]); echo(" º W"); ?></td>
</tr>
<tr>
<td>BBOX maxY:</td>
<td align="center"><?php echo($BBOX[3]); echo(" º W"); ?></td>
</tr>
<tr bgcolor="#CFCFCF">
<td>ResX:</td>
<td align="center"><?php echo($_GET["RESX"]); ?></td>
</tr>
<tr>
<td>ResY:</td>
<td align="center"><?php echo("-"); echo($_GET["RESY"]); ?></td>
</tr>
<tr bgcolor="#CFCFCF">
<td>Format:</td>
<td align="center"><?php echo($_GET["FORMAT"]); ?></td>
</tr>
</tbody>
</table>


<?php
//checking the service
if (strnatcasecmp ($_GET["service"],'wcs') == 0) {

//start of form
echo('<FORM action="ows.php" method="GET"><input type="hidden" name="service" value="WCS"><input type="hidden" name="request" value="getCoverage"><input type="hidden" name="coverage" value="');echo($OWS_name);echo('">');

//mid section
echo('<input type="hidden" name="CRS" value="');echo($_GET["CRS"]);echo('">');
echo('<input type="hidden" name="BBOX" value="');echo($_GET["BBOX"]);echo('">');
echo('<input type="hidden" name="RESX" value="');echo($_GET["RESX"]);echo('">');
echo('<input type="hidden" name="RESY" value="');echo($_GET["RESY"]);echo('">');
echo('<input type="hidden" name="FORMAT" value="');echo($_GET["FORMAT"]);echo('">');

//final secion
echo('<p align="center"><INPUT type="submit" value="Request Coverage"></p></FORM>');

}
?>

OWS_request

2007-04-20T17:41:00.000+01:00

The Legend is static and is drawn at the start up so I couldn't make a link in the download icon making a specific WxS request.

The solution was a javascript function that is called when the icon is pressed. This function will get the service and the name of the layer, later it will request adicional information (getDescription) so that a proper WxC can be performed.

After getting all the information the functions opens a window and calls a PHP file (passing by GET all the information for the WxS). The new pop-up will have all the values hidden inside a form. When the user click on "submit" the final WxS will be performed. The pop-up also containes a table with all the information on the request

function OWS_request (OWS_name, OWS_service) {
//example of request
//ows.php?service=srtm10x&request=getCoverage&COVERAGE=srtm10x+&CRS=epgs:4326&BBOX=41,75,45,50&RESX=0.00833&RESY=0.00833&FORMAT=GEOTIFFINT16';

//Creation of the XTMLHTTP object and request of information coverage
request=new getXMLHTTP()
var URL='ows.php?service=WCS&request=describeCoverage&coverage='+OWS_name;
request.open ('GET',URL,false) ; //wait for response
request.send(null);
var xmldoc=request.responseXML;
//necessary info: CRS,RESX,RESY,FORMAT,BBOX the name is alreay given
//the XML programming was from the AJAX for Dummies
//for example the tag gml:offsetVector has the name offsetVector the GML is nothing
var CRSnode=xmldoc.getElementsByTagName("requestResponseCRSs");
var CRS=CRSnode[0].firstChild.nodeValue;
var RESnode=xmldoc.getElementsByTagName("offsetVector");
var sRESX=RESnode[0].firstChild.nodeValue;
var RESX=sRESX.split(" "); //array with resx in this case the first value is the important one
var sRESY=RESnode[1].firstChild.nodeValue;
var RESY=sRESY.split(" ");
var FORMATnode=xmldoc.getElementsByTagName("formats");
var FORMAT=FORMATnode[0].firstChild.nodeValue;

/*The BBOX comes from the getGeoExtents, were it is defined that
gminx=minx;
gmaxx=maxx;
gmaxy=maxy;
gminy=miny;

These are golbal variables, so no problem to access them, also they are always changed according to the events of kamap

BBOX = lower left (min x, min y) and upper right (max x, max y)
*/
var BBOX = new Array(gminx,gminy,gmaxx,gmaxy);

//creation of the URL that passes the information to the popup.php
//note: RESY fix in the string (it canot be negative)
var url_str='popup.php?service='+OWS_service+'&request=getCoverage&COVERAGE='+OWS_name+'&CRS='+CRS+'&BBOX='+BBOX.join(",")+'&RESX='+RESX[0]+'&RESY='+(-1*parseFloat(RESY[1]))+'&FORMAT='+FORMAT;

//open window
var window_popup=window.open(url_str,null,"height=300,width=375");
};

Layer's HTML

2007-04-20T17:37:00.000+01:00

The HTML that generate the Legend is in member createLayerHTML of the Kalegend.

After the HTML for the query control I added the following if (a bit similar to the query one

if (this.showDownloadControl) {
td = document.createElement('td');
td.width = '14';
if (oLayer.isDownload()) {
a = document.createElement('a');
a.href ='javascript:OWS_request(\"'+oLayer.name+'\",\"'+oLayer.ows_service+'\")';
img = document.createElement( 'img' );
img.width = '14';
img.height = '14';
img.src = 'images/download_ok.png';
img.border='0';
img.id=oLayer.name;
img.alt = "OWS download request";
img.title = "OWS download request";
a.appendChild(img);
td = document.createElement( 'td' );
td.appendChild(a);

} else {
img = document.createElement( 'img' );
img.width = '14';
img.height = '14';
img.src = 'images/download_no.png';
img.oLayer = oLayer;
img.id=oLayer.name;
img.alt = "OWS not available";
img.title = "OWS not available";
td = document.createElement( 'td' );
td.appendChild(img);
}
td.width = '16';
tr.appendChild(td);
}

This code will simpely check if the layer can/can't be downloaded and give the specific icon

WCS and WFS metadata

2007-04-16T11:21:00.000+01:00

I forgot that aside from the information if the map is downloadable or not is is also necessary to know what service, so another metadata tag was added

METADATA
"ows_service" "wcs"
END

it can be "wcs" or "wfs"

The programming is basically the same for the download but simpler since is it only a property and it doesn't need functions like isDownload()

The getMetadata is a bit different because the wcs and wfs are string and need '', otherwise the javascript eval() will not work (this functions runs the init.php reply)

$ows_service = "null";
if ($oLayer->getMetaData ("ows_service") !="") {
if(strcasecmp($oLayer->getMetaData("ows_service"), "wcs") == 0)
$ows_service = "'wcs'";

if(strcasecmp($oLayer->getMetaData("ows_service"), "wfs") == 0)
$ows_service = "'wfs'";

}

Let there be KaLEGEND

2007-04-15T16:34:00.000+01:00

To start the KaLegend object has its options defined in the myOnload() function of the StartUp.js (which is located in the /tools/kaAqua path....)

I defined the option of download like the other properties:

legendOptions.download = typeof gbLegendDownloadControl != 'undefined' ? gbLegendDownloadControl : true;

the gbLegendDownloadControl is not necessary, it is just here to look good :)

jumping to the kaLengend constructor in the kaLegend.js, It was added a new property:

this.showDownloadControl=true;

and some codelines lower:

this.showDownloadControl = typeof options.download != 'undefined' ? options.download : true;

the standard options is to show the controls unless otherwise defined in the options.download

_map / _layer object and new methods/properties

2007-04-15T15:37:00.000+01:00

The previous post passed the metadadata to a php variable that contains the properties of the layers, later these values are passed to the object it self when is created by the constructor funcion.

The layer object contains several important methods and properties that will be used mainly by kaLegend to draw everything that is needed to download the layer

The layer object is defined in the kaMap.js and in the "contructor" function (is it correct to call it constructor ?! ) It was added:

this.download = (typeof(o.download) != 'undefined') ? o.download : false;

This is necessary to be certain that the layer always has this property as true or false

The next step is to create a member that returns is a layer is donwloable or not

_layer.prototype.isDownload = function() {
return this.download;
}

the member isDownload will be usefull in the _map object, since this object need a method that gives the list of downloadable layers

_map.prototype.getDownloadLayers = function() {
var r = [];
var l = this.getLayers();
for( var i=0; i<l.length; i++) {
if (l[i].isDownload()) {
r.push(l[i]);
}
}
return r;
};

Fetching Metadata

2007-04-15T15:24:00.000+01:00

THe metadata contained in the MAP file is fetched at init.php and based on the programming already there:

$download = "false";
if ($oLayer->getMetaData ("download") !="") {
if(strcasecmp($oLayer->getMetaData("download"), "true") == 0)
$download = "true";
}

This will get the metadata from dowload in the MAP file and pass the value to the $download variable of PHP, as default the $download is false

Then the $download is integrated in the layer as another property:

$szLayers .= "map.addLayer(new _layer( { ".
"name:'".$groupName."',".
"visible:".$status.",".
"opacity:".$opacity.",".
"imageformat:'".$imageformat."',".
"queryable:".$szQueryable.",".
"download:".$download.",".
"tileSource:'".$tileSource."',".
"redrawInterval:".$redrawInterval.",".
"refreshInterval:".$refreshInterval.",".
"scales: new Array('".implode("','",$groupScaleVis)."')}));";

The line in bold was added to the properties of the layer

New MAP metadata

2007-04-15T15:15:00.000+01:00

I hace decided to use the same strategy of "is the layer queryable" but this time to the WCS so that users can download the layers, by simplely clicking in a small icon

The first thing is that information concerning the layer is has to be indicated in the MAP file as metadata and then retrived by kamap

Now the MAP file to have a WCS layer downloaded by the Kamap needs the following metadata

METADATA
"download" "true"
END

segmentation fault in WCS

2007-04-08T20:41:00.000+01:00

I was going thru the WCS and started to have segmentation faults and everything was crashing,
finally I discovered the problem

The RESX and RESY are postive numbers, despite sometimes RESY being given as a negative value.

The funny thing was the PID was crashing and the sript was sending a ows.php reply, even more funny is that wcs_resolution is given with Y as negative value, at least in all the examples that I saw

Also I read the documentation and it says that BBOX is required, well actually not !!! without the BBOX it will reply with all the layer.

WCS headache

2007-04-07T19:02:00.000+01:00

Well Today I had a huge headache

The big problem started when I was doing a DescribeCoverage request and was getting nothing, as if there was no WCS layers (I had all the WCS metadata inside the map file), the result was only this

<CoverageDescription version="1.0.0" updateSequence="0" xsi:schemaLocation="http://www.opengis.net/wcs http://schemas.opengis.net/wcs/1.0.0/describeCoverage.xsd">
</CoverageDescription>

So no error, no major problems and of course nothing on the net!!!

I tryed to make a getCoverage and got an error message concerning projection,

Since my server is jailed there was no epsg file for the mapserver to read, I never had a problem with projections because I never had the need for epsg, since my projection was like this:

PROJECTION
"proj=latlong"
"ellps=WGS84"
"datum=WGS84"
END

(no: "init=epsg:4326")

So copying the epsg file to the chroot and adding the location of it to the map file solved the problem (CONFIG "PROJ_LIB" "/usr/local/share/proj")

Then I got a nice reply fron the DescribeCoverage, still I got some problems with the GetCoverage because instead of epsg I had epgs (....I hate this type of mistakes)

It took me several hours to discover all the problems....I wish that a better error log system would be in place for WxS

HTTP Header

2007-04-07T12:30:00.000+01:00

This makes the HTTPS for a tiff file with name srtm-10x:

header('Content-type: image/tif');
header('Content-Disposition: attachment; filename="srtm-10x.tiff"');

WEP Cracking

2007-04-06T09:58:00.000+01:00

I can't belive that a WEP key can be cracked in 60 seconds

http://blogs.zdnet.com/hardware/?p=347

It took me 3 weeks to crack my neighboors WEP using Kismet!!!! To discover that he had a stupid password.

Maybe I should use WAP instead.

WCS wrapper for Mapserver

2007-04-05T16:04:00.000+01:00

WMS,WFS and WCS is how it is possible to connected to a webGIS server and to download data:
WMS == Raster as an image
WFS == Vectorial information
WCS == Raster as a layer (original raster with values)

The request of WxF is normally done by a GET to the Mapserv program (is like a normal CGI-BIN for program execution), well I don't want Apache to run any program, so the solution is a wrapper PHP script, where the script catches the requests and uses them for a valid WCS reply.

The wrapper is very simple and there is a basic example on the mapserver home page, This is a wrapper with small modifications to work with WCS:

<?php
/*ows.php?service=WCS&version=1.0.0&Request=GetCapabilities*/
$request = ms_newowsrequestobj();

//$request->loadparams();

foreach ($_GET as $k=>$v) {
$request->setParameter($k, $v);
}

// forcing to version 1.0.0
$request->setParameter("VeRsIoN","1.0.0");

ms_ioinstallstdouttobuffer();

$oMap = ms_newMapobj("/usr/local/apache2/htdocs/secure/ms/data/calter.map");
$oMap->owsdispatch($request);
$contenttype = ms_iostripstdoutbuffercontenttype();
$buffer = ms_iogetstdoutbufferstring();

header('Content-type: application/xml');
echo $buffer;

ms_ioresethandlers();

?>

For start Mapserver only suports WCS version 1.0.0 so any request for higher versions has to be redirect:
$request->setParameter("VeRsIoN","1.0.0");

There seems to be a bug with $request->loadparams(); this produces an empy file reply from the server with no errors, so the solution it to get the parameters directely from the GET
foreach ($_GET as $k=>$v) {
$request->setParameter($k, $v);
}

if replacing $_GET for $_POST is should be possible to use POST to request the WCS, mapserver only accepts GET.

Email hidding

2007-04-05T15:45:00.000+01:00

I did a small info page concerning the webGIS, it is a page with a semi-transparent background, like the KaExplorer example, but ported to KaAqua. Anyway I had to put some email and as usual I have hidden the email using a common javascript procedure, where the email is transformed in to an array of number and then the documment.write() reconverts back to letter.

http://www.dynamicdrive.com/emailriddler/

For my big surprise, when I make the page appear inside a div tag the documment.write() doesn't work , and there is no errors not in the javascript console or in the apache.

I have no idea why or even found the reason why, in the end I had to hide the email using simple echo() in PHP

Everything is now locked !!!!

2007-04-05T14:23:00.000+01:00

Well the acesses to the mapserver is now locked !!!!!!!

For locking the system with a login I used the PHP classes called adminpro that I got from the site:

http://www.phpclasses.org/

The programming was done by Giorgios from Greece, and is the most straight forward login system based on MySQL, that I know, all programming is simple to follow and the configuration is very simple.

I had some problems making the socket connection to the MySQL, I don't understand why the socket wouldn't work, the file was inside chroot but I was having error messages saying no socket.

I couln't find much about the problem on the net, so instead of socket I used a direct connection: 127.0.0.1:3306

I don't know if this is secure but it worked !!!

<?php include("adminpro_class.php");
$prot=new protect();
if ($prot->showPage) {
?>


<?php } ?>

Cool and simple !!!

Ka-maps instalation

2007-04-05T13:37:00.000+01:00

The installation of ka-maps was simple and the only problems was the paths to the calter.map and the config.php paths, I had to change several javascripts files but in the end everything was working fine.

The KaAqua was my favourite and it looks cool with the transparent stuff, I also love the legend box and how you can close/open it

Why not Openlayers ?

2007-04-05T13:32:00.000+01:00

Despite the look of Ka-maps, there is the problem of Ka-maps not having an API and at first the code looked a bit confusing.

So I have a look at Openlayers, they seem to be more organized than Ka-maps but I didnt liked that the server requests are based on WxS and there is no PHP as "back-office", never the less the options given like the connection to google and virtual earth extremely interresting.

So in the end I started to work with Ka-maps

Why not Ka-map

2007-04-05T13:19:00.000+01:00

Since chameleon was a disaster I then decided to give a look at ka-maps

I found a nice tutorial on setting up ka-maps/AJAX

http://www.xml.com/pub/a/2005/08/10/ka-map.html

Also the page from ominiverdi is kick ass because it was a wiki and several examples

http://www.ominiverdi.org/

Why not Chameleon

2007-04-05T13:05:00.000+01:00

I can't belive that I haven't posted anything for one month !!!!! Despite one week vacation things have progressed.

I did some research on chameleon and how the tools/widgets from chameleon could help me. Chameleon is done by DM solutions and basically you put tags on your HTML/PHP and it gets replaced by a widget, the concept is extremely cool and efficient, I think cold fusion works in the same way?

The problem is that I couldn't make it work I spent some days fighting it and there was always problems with paths and configuration files, even the tools that should help you work with chameleon were messy and the documentation was not very helpfull

The instalation of chameleon seems to be straight forward if you use a special installation pack called FGS is seems even better

Well my mapserver is chrooted and the apache is not compiled with Alias module (due to security concerns) so in the end it was extremely problematic trying to install chameleon in the server.

So I gave up

EXTENT versus SIZE in Mapserver

2007-03-04T17:02:00.000Z

I am working with 3 vectorial layers and these layers are bigger than the research area (EXTENT) I have defined the extent area to be 35 30 95 60 (lower left corner, upper right corner) dd of lat/long. Still the mapserver was rendering data from outside the extent area

It took me a while to understand the reason. The problem is with the SIZE parameter, in the above pic the size was 800 500 so the area was a bit out of the EXTENT and Mapserver will not have a problem to render any data outside the EXTENT so that the image gets the proper size.

Changing the SIZE to 800 400 I get a prefect image

Let there be color!!!

2007-03-04T15:46:00.000Z

I never liked the DEM of the SRTM to be in a 16bit grey scale, so I searched for a solution. Normally in GRASS you select the SRTM color rules for it, but in Mapserver I couln't find a good enough answer on how to deal with colors in DEM (it is possible to make color scales in satellite images using the band, but a BEM only has "one band").

I have found this technical blog http://www.perrygeo.net/wordpress/?p=7 that has some nice tools from DEM processing, the tools are in C++ and need the GDAL library to be compiled.

One tool is the color-relif (there are also tools to make hillshades, slopes and aspect). So I just compiled the programs, went to /usr/local/grass-6.2.1/etc/colors to get the srtm color file, which indicates the RGB values according to the altitude. The GRASS file has sometimes instead of RGB values a color name, so in the SRTM colors there was "aqua" for 0 mt altitude, so I just changed to 0 0 255 (total blue). The
final color table is something like this

-500 0 0 10
-300 0 0 20
-200 0 0 70
-100 0 0 130
-50 0 0 205
0 0 0 255
0.1 57 151 105
100 117 194 93
200 230 230 128
500 202 158 75
1000 214 187 98
2000 185 154 100
3000 220 220 220
5000 250 250 250
8850 255 255 255

1st column is altitude and the other columns the RGB values.
So the DEM that was almost totally dark in GIMP because a very nice colored map.

The program simplely creates a band2 and 3 from the original band1 and assigns a diferent color value acording to the color table. I had the SRTM file in a tiled structure, after running the program the final file was as orignally (without internal tiles)

aemp2007

2007-02-28T19:17:00.000Z

Just finished the second version of the aemp2007 congress web page. This time it looks nice, with some nice CSS menus that I got from here and the javascript image blending system was from here.

FATAL ERROR: Session save path () doesn't exist or is not writable

2007-02-21T15:34:00.000Z

OK. I decided to work with the DM tools and chamelion widgets for the CALTER mapserver, this way I will not have to program any basic components.

I am still installing all the stuff, but to have chamelion working the PHP moduled needed to re recomplied to support GD and FreeType

./configure --enable-safe-mode --with-mysql --with-pgsql --enable-dbase --with-config-file --enable-static --prefix=/chroot/httpd/usr/local/apache2/php --with-apxs2=/chroot/httpd/usr/local/apache2/bin/apxs --disable-cgi --with-config-file-path=/chroot/httpd/usr/local/apache2/php --with-openssl --with-zlib --with-gd --with-freetype-dir

The maplab tools gave some headaches with Forbidden warnings. The fist time this warning was caused incorrect permissions (nothing like a good 666 or 777 to make things working).

Still it wouldn't work, checking the error logs it was mod_security sending the forbidden, the php was sending an error message and mod_security was blocking it. So the mod_security was reconfigured not to scan replies from the server

Finally the error was FATAL ERROR: Session save path () doesn't exist or is not writable , the maplab uses sessions id to pass data between the programs, so I googled and found the solution in a french website.

The php.ini needs to have session.save_path unquoated and activated to /tmp and /tmp should have rw permission.

2nd CISTI and XSS

2007-02-16T08:24:00.000Z

Just submmited a 8 page paper for the CISTI (2nd Iberian Conference on Information Systems and Technologies). The paper is called "Security enhancement in a WebGIS application". Basically it describes Chroot, Firewalls (IPtables), Injection prevention of tags/scripts, HTTPS redirection and error message blocking, in a Mapserver system.

Yesterday I was reading on the tech news that 7 out of 10 sites allow XSS attacks (cross-site scripting) were you can for example send SQL tags in the URL (GET) and do some real shit.

Error in Tags

2007-02-16T08:21:00.000Z

Yesterday I couldn't find the script to download the SRTM in my computer and copy/pasted what I had in here, it seems there was problem. When the script was posted the blog system made a confusion with the STDIN command that is in "<" and ">" this was fixed using special html code for Special Characteres.

Apache compilation (security options)

2007-02-06T17:54:00.000Z

Some months ago I had to compile the Apache server for the CALTER webGIS and it was a bit tricky because it needed an external module called mod_security and the version 2.0 refused to work (come GCC 4.1 problem if I recall). This is a some how compilation with security on mind

./configure --prefix=/usr/local/apache2 --with-mpm=prefork --disable-charset-lite --disable-include --disable-env --enable-setenvif --disable-autoindex --disable-asis --disable-cgi --disable-negotiation --disable-imap --disable-actions --disable-userdir --disable-alias --disable-so --enable-ssl --enable-modules=acess --enable-module=log_config --enable-module=dir --enable-module=auth --enable-unique_id --enable-usertrack --enable-proxy --enable-proxy-http --with-module=prox y:modules/proxy/modsecurity-apache_1.9.4/apache2/mod_security.c --enable-securi ty --enable-so --enable-rewrite

The setenvif should had been disable, but it in the end was necessary, If I recall it was PHP that was in need.
The mod_security has to be unpacked to the directory /modules/proxy and also the the Apache needs to be compile with the --enable-proxy-http

The modules are all statically linked, initially I wanted everything statically so there was no load modules options but I didn't managed to statically compile PHP in Apache2.

xtmas, new year and classes

2007-02-02T16:08:00.000Z

Things have been a bit down concerning the blog, due to all the holidays and the GIS classes that I gave in the University of Huelva.

I had to study ArcGIS to give the classes, last time I used anything from ESRI was 4 years ago, the experience was excellent and the students helped a lot ....

Problems with UALG's proxy (of the SRTM script)

2006-12-11T18:06:00.000Z

In BGU the proxy accepts FTP connections and everything runs ok (as long the Iptables is off and the enviromment has the proxy's url). The problem is that the proxy from UALG only accepts HTTP, I have tried to upgrade the FTP connection of the script to deal with it, but it was a nightmare.

The simplest solution was to use the wget command instead of an FTP connection

The the case of running from BGU or UALG the enviromments need to have the proxy:
BGU: http://wwwbgu.ac.il:8080
UALG: http://proxy1.si.ualg.pt:8080

#!/usr/bin/perl

use strict;
use warnings;
use Net::FTP;
#variable declaration
#specific varibles, change here according to needs
my $ftp_url="ftplmu.jrc.it";
my $ftp_pwd="/pub/SRTM_v3/SRTM_Data_GeoTiff";
my $ftp_file;
my $cmd;
my $ftp;
#input variables
my $max_x;
my $min_x;
my $max_y;
my $min_y;
#other variables
my $temp_min_y;
my $formated_number_y;
my $formated_number_x;
my $file_url;
#user input
print "Enter tile number of maximum long. (x): ";
$max_x=<STDIN>;
chomp($max_x);
print "Enter tile number of minimum long. (x): ";
$min_x=<STDIN>;
chomp($min_x);
print "Enter tile number of maximum lat. (y): ";
$max_y=<STDIN>;
chomp($max_y);
print "Enter tile number of minimum lat. (y): ";
$min_y=<STDIN>;
chomp($min_y);
#for loop to generate the url and open the FTP connection
$temp_min_y=$min_y;
for ( ;$min_x<=$max_x; $min_x++) { #some nice C/C++ sytanx $formated_number_x=sprintf("%02d",$min_x); $min_y=$temp_min_y; for( ;$min_y<=$max_y; $min_y++) { $formated_number_y=sprintf("%02d",$min_y); $ftp_file="srtm_".$formated_number_x."_".$formated_number_y.".zip"; #using wget to get the file, wget will use the system's variable of proxy $cmd="wget ftp://".$ftp_url.$ftp_pwd."/".$ftp_file; system($cmd); } }

Downloading the SRTM V3.0 using a PERL script

2006-12-01T19:50:00.000Z

GeoTiff is the best format to download because it runs very well with GDAL and GRASS, the tiles are in 5º by 5º. I am interrested in area between 35ºE -> 100º E; 30ºN -> 60ºN, this is a huge square from the western med. till the middle of China. The main objective is only the Aral Sea, but since I need to catch all the NIS countries the area has to be relatively big (Kazakhstan is HUGE!!!)

From the map I need all the tiles from x=44 y=6 till x=56 y=1 (left bottom till rigth top) this gives a big amount of clicks/downloads !!!! Using the approach of lets spend 99% of the time making a script that will do 100% of the job in 1% of the time, I decided to make a simple PERL scrip.

I am no PERL expert, I normally program in C/C++ so I am not familiar with PERL, but it was relatively simple because I had a book around the house called "Perl for C programmers" and the code lines for the FTP access where more or less copied from the explanation of how the Net::FTP module works.

The tile files have a structure like srtm_X_Y.zip so the script asks for the minimal and maximal number for x and y, after it makes the URL string and opens an FTP connection to the italian server. The script only has the debug dumping of the FTP connection.

Just copy/paste the next files to a file and run it:

#!/usr/bin/perl
#The script generates the URL of the files to download
#then it opens an FTP connection a gets the file
#simple, clean without much programming advances and it works
#note: the file code is something like this srtm_X_Y.zip
use strict;
use warnings;
use Net::FTP;
#variable declaration
#specific varibles, change here according to needs
my $ftp_url="ftplmu.jrc.it";
my $ftp_pwd="/pub/SRTM_v3/SRTM_Data_GeoTiff";
my $ftp_file;
my $ftp;
#input variables
my $max_x;
my $min_x;
my $max_y;
my $min_y;
#other variables
my $temp_min_y;
my $formated_number_y;
my $formated_number_x;
my $file_url;
#user input
print "Enter tile number of maximum long. (x): ";
$max_x=<STDIN>;
chomp($max_x);
print "Enter tile number of minimum long. (x): ";
$min_x=<STDIN>;
chomp($min_x);
print "Enter tile number of maximum lat. (y): ";
$max_y=<STDIN>;
chomp($max_y);
print "Enter tile number of minimum lat. (y): ";
$min_y=<STDIN>;
chomp($min_y);
#for loop to generate the url and open the FTP connection
$temp_min_y=$min_y;
for ( ;$min_x<=$max_x; $min_x++) { #some nice C/C++ sytanx $formated_number_x=sprintf("%02d",$min_x); $min_y=$temp_min_y; for( ;$min_y<=$max_y; $min_y++) { $formated_number_y=sprintf("%02d",$min_y); $ftp_file="srtm_".$formated_number_x."_".$formated_number_y.".zip"; #creating the FTP object and getting the file $ftp = Net::FTP->new($ftp_url, Debug=>1, Passive=>1) or die "It is not connecting: $@";
$ftp->login("anonymous");
$ftp->cwd($ftp_pwd);
$ftp->message;
$ftp->binary;
$ftp->get($ftp_file) or die "Get failed ", $ftp->message;
$ftp->quit;
}
}

Getting SRTM data (version 3)

2006-12-01T16:17:00.000Z

The SRTM data is the altimetric data created by the Shuttle Radar Topography Mission, and it is available in a resolution of 3-arc second (90meter) outside the USA. Version 3.0 has finshed SRTM data, DEM void's filled, clips of the coastlines.

Methodology of SRTM V3.0

WebGIS to download tiles of SRTM V3.0

This server has the option of mirror picking and a direct access to the FTP

I tested the 3 mirrors for velocity of download the srtm_48_05.zip file, my location is Portugal and my ISP says that my ADSL is at 24mbits (Ya shore......) and the test was at 4:54PM (Lisbon Time) in 29 Nov. 2006
HTTP connection of CSI server: 268kB/sec
FTP connection of CSI server: 154kB/sec

HTTP connection of JRC(IT) server: 254kB/sec
FTP connection of JRC(IT) server: 470kB/sec

HTTP connection of Kings College (UK) server: 118 kB/sec
FTP connection of Kings College (UK) server: 351 kB/sec

So the velocity doesn't change much but it seems (at least for me) that the Italian server is better for my location

Also there are some FTP servers with version 2.0 and 1.0:
ftp for version 1.0 and 2.0

After GRASS was installed.....

2006-11-30T14:44:00.000Z

After GRASS is compiled the GDAL need to be recompiled pointing to the GRASS directory
>./configure –with-grass=/usr/local/grass-6.2.0 && make

Normally when using gdal (in this case gdalinfo) it may get an error message like this

>gdalinfo: error while loading shared libraries: libgrass_vect.so: cannot open shared object file: No such file or directory

This means that gdal is not finding the libraries of GRASS, the best thing is to put the GRASS path on the ld.so.config and run the ldconfig

>echo /usr/local/grass-6.2.0/lib >> /etc/ld.so.conf
>ldconfig

If you put > instead of >> it will fuck up all ld.so.conf

Complete ./configure options for GRASS 6.2

2006-11-30T14:38:00.000Z

The new GRASS 6.2 has been out for sometime and normally when I have to compile GRASS I need to type a HHUUUUGGEEEE command line to have everything that I need working.

Normally I put the command line in a file called setup.sh and then run it

./configure --with-postgres-includes=/usr/include/pgsql --with-cxx --with-mysql --with-mysql-includes=/usr/include/mysql --with-ffmpeg --with-ffmpeg-includes=/usr/local/include/ffmpeg –enable-largefile --with-readline --with-sqlite --with-gdal=/usr/local/bin/gdal-config

This works for my linux box (suse 10.1) and it will give

- Access to MySQL and PostGIS

- Create movies (mpeg)

- Use GDAL to acess files (after GRASS is intalled GDAL need to be recompiled)

- Enable readline (to go back on the commands)

- Things like enable-largefile and enable-cxx just look good :)

Introduction

2006-11-30T13:58:00.000Z

To start a blog a person should introduce him self: Born in the sleepy town of Tavira in the shores of the Gilão river in the Algarve, southern most region of Portugal, learned Agriculture in the Algarve University and then moved to Israel to Ben-Gurion University for 4 years to make a Ph.D. in application of Geostatistics and Principal Component Analysis in Organic Agriculture.

In the middle of the master and Ph.D I started to work in GIS and since I am a big linux fan/user it was it was natural for me to adopt GRASS as my GIS platform and to work in R as my mathematical “calculator”

In this next year I'll be working in a project where the Portuguese team will have to make a webgis system meaning 1 year of grass and mapserver !!

This blog will help me to keep track of some of my work and it will also be a place for me to write my notes, because my memory is very crapy and I can only recall 1% of all the grass commands and HOWTOs