gEt DoWn ToNiGhT » Programming

Sample Page for sending JSON data as string to web-service

Sam — Mon, 05 Dec 2011 16:56:04 +0000

Just realised it has been a year since I posted anything on here! Time really flies when you’re busy!

I remember when I was first starting off with jQuery, JSON and all that jazz that I struggled to find some simple examples of how to do things like send data to a web-service.

Here is a quick example of how to send some JSON data (as a single post field) to a web service, using jQuery’s post method.

Keep in mind that if you are posting to another domain you may need to use something called jsonP – but if you are just learning, this is a good starting point!



    
        JSON Web Service Demo
 
        
 
        
 
        <script type="text/javascript">
            $(document).ready(function() {
 
                //this code will be added to the onclick event
                $('#save_user_button').click(function() {
 
                    //add the values we need for the API to an object
                    var data = new Object;
 
                    //build the object
                    data.guid                   = $('#guid').val();
                    data.defaultallocation      = $('#projecttype').val();
                    data.email                  = $('#email').val();
                    data.username               = $('#username').val();
                    data.fullname               = $('#fullname').val();
 
                    //turn the object in to a JSON string
                    var myJson = JSON.stringify(data);
 
                    var url = "http://testurl/service/user";
 
                    //call the post
                    $.post(url, {jsonUser : myJson}, function(myResult) {
                        alert(myResult)
                    });
 
                });
 
            });
        script>
 
    
    
        
            
 
                
 
                
 
                
 
                
 
                
 
            
                    Project Type
                    
                

                    GUID
                    
                

                    User Name
                    
                

                    Full Name
                    
                

                    Email

Slow and Locked MySQL Queries

Sam — Fri, 17 Dec 2010 17:08:59 +0000

One of our clients’ servers recently saw a huge drop in speed. None of the hardware had been changed, and while the total number of users has been steadily increasing for the last few years, it has not seen any kind of spike that I thought might cause this type of problem.

In order to try to find the fault or problem, I had to look in many different places and discovered a lot about Linux and MySQL. I enjoy these fact finding missions, even though there are always a lot of dead ends.

I thought I would use this blog as a place to write down all the areas that are worth checking, in case anyone else finds themselves in this situation.

Things to check:

The Process List
MySQL Slow Query Log
Server free disk space
Server memory usage
Server load
Your queries
Your indexes

Our server was running on Linux Centos 3, and many of the commands below are Linux only. For executing the MySQL commands and queries I suggest getting something like WebYog’s excellent SQLYog, or using MySQL’s own MySQL Workbench.

Listing active MySQL queries

MySQL contains the ability to show a list of currently running queries. From within MySQL – execute the following query:

SHOW PROCESSLIST;

This will show a list that details which users, hosts, databases are being used to execute queries – how long they have been going for, and what state they are in. Look for queries that have been running for a long time, or have the state of ‘locked’. For more information see here.

Breaking down your queries

It is crucial that you enable slow query logging and it is quite simple to set up. Edit your my.cnf file, and make sure the following is present:

log-slow-queries = /var/log/mysql/mysql-slow.log
long_query_time = 3

(Adjust the log_query_time setting to suit the length of the query you are trying to detect)

Once this is in place, restart the MySQL server and start checking the new log file.

The MySQL command EXPLAIN allows you to see a breakdown of the steps taken by MySQL to get your query results, including information about joins and the sequence of queries called. You can use the information it outputs to help optimise your own queries.

The utility is easy to use. Simply prefix your SELECT statement with the word EXPLAIN. eg.

EXPLAIN
SELECT userName, firstName, lastName
FROM users
WHERE firstName LIKE 'da%';

top

This command shows you a list of applications and services running on the machine, along with some basic information about the resources they are using.

From here you can see the system load, free memory and the % of CPU for each process.

To quit and return to the shell, use CTRL+C.

df -h

This will show you how much free space is on your mounted drives, in human readable form.

[sam@webserver1 /]$ df -h
Filesystem            Size  Used Avail Use% Mounted on
/dev/md1              9.8G  3.9G  5.9G  40% /
none                 1014M     0 1014M   0% /dev/shm
/dev/md2               57G   31G   27G  54% /disk1

tail

This is a quick way of looking at the last few lines of a log file. The following will show you the last 10 lines of the mysql log.

tail /var/log/mysql.log

You can also specify that you want to view a specific number of lines by using the -n parameter. eg.

tail -n 50 /var/log/mysql.log

Eventually, after going through lots of checking and tests, we discovered that the server was simply at capacity. It hadn’t been upgraded for 8 years and the number of users had quadrupled during that time – with users from 2 other continents now logging on regularly. The database was on the same hardware as the web-server software and the system was just becoming overloaded.

We managed to speed things up considerably before we were forced to upgrade, mostly by examining the slow-query-log and adding indexes where appropriate. Also, many of our queries were refined and tidied up – so we were definitely not wasting our time running the examinations. It has made me realise that it is probably worth going back over old code every now and then to check for potential improvements – something we only tend to do if there is a client requirement, or an OS upgrade required.

We have now performed a complete upgrade to the latest hardware available, with 8GB of memory, more CPU cores than I suspect are really needed, the latest Centos 5 OS and all the most recent MySQL/Apache and PHP versions – which has made the system run incredibly quickly.

Using the database from inside a helper (CodeIgniter)

Sam — Thu, 17 Jun 2010 09:06:17 +0000

I have some custom functions that I store in a helper in CodeIgniter.

Some of those functions need to use the database, and to do so you have to explicitly tell the function to load the Code Igniter framework and database class.

Its pretty simple when you know how, but it took quite a while to work it out.

function get_user_full_name($userId) {
 
    //the database functions can not be called from within the helper
    //so we have to explicitly load the functions we need in to an object
    //that I will call ci. then we use that to access the regular stuff.
    $ci=& get_instance();
    $ci->load->database();
 
    //select the required fields from the database
    $ci->db->select('firstName, lastName');
 
    //tell the db class the criteria
    $ci->db->where('userId', $userId);
 
    //supply the table name and get the data
    $query = $ci->db->get('user');
 
    //ensure that there is something in the variable - in case of no match
    $fullName = "";
 
    foreach($query->result() as $row):
 
        //get the full name by concatinating the first and last names
        $fullName = $row->firstName . " " . $row->lastName;
 
    endforeach;
 
    // return the full name;
    return $fullName;
 
}

Using Eclipse as a PHP IDE in Ubuntu 9.10

Sam — Wed, 25 Nov 2009 17:05:40 +0000

This is actually much easier than any of the guides I found online make out.

Firstly, install Eclipse from the repositories. Then open it up, select your default workspace (if you havent already) and then:

From the Help menu, select “Install new software”.
Click the “Add” button and add the following:
- Name: Eclipse Updates (Galileo)
- Location: http://download.eclipse.org/releases/galileo/
Apply your changes and wait for the contents to download.
When all is ready, select your new source from the drop down and look for the “Programming languages” section.
Open it up and add the “PHP Development Tools (PDT) SDK Feature”.
Step through using Next and Finish etc, and after a few moments downloading and installing you now have a PHP ready IDE.

Loading JSON data – the EASY way!

Sam — Wed, 18 Nov 2009 12:22:04 +0000

A few weeks ago I wrote a blog post about using jQuery and JSON to automatically load data from a JSON file in to name-matched controls on an HTML page.

It is only now that I realise quite how powerful this is!

The great thing about it is that you do not have to edit your data-load function if you add a new field to the database. Because the jQuery function loops through all the fields in the JSON data, it will automatically insert the values in to the correct fields, providing those fields are named the same as the html form controls.

Here is the code again for those who dont want to check back.

$.getJSON("/yourJson.js, function(myJson){
	$.each(myJson.rows, function(i,item) {
		 for (prop in item) {
		 	 $('#' + prop).val(item[prop]);
		 }
	});
});

Installing MyODBC 3.51 on Windows 7 x64

Sam — Mon, 16 Nov 2009 11:11:44 +0000

In order to get some of our 32 bit applications working on Windows 7 64 bit edition, we need to provide support for the MySQL 3.51 ODBC connector.

This is actually quite simple to do, as long as you know what to look for. The ODBC manager that you are able to access from within Windows is not going to work for you, but it turns out that Microsoft provide a 32 bit manager that will.

To use this manager, run the following command:

c:\windows\syswow64\odbcad32.exe

Please note that it is not good enough to just type odbcad32.exe, as there is another file in the path with the same file name (great work MS!). However, once you have opened this application up, it works in the exact same way as if you were running XP.

Setting up a basic web server using Ubuntu 9.10 (Desktop)

Sam — Thu, 12 Nov 2009 17:14:08 +0000

I have been using Ubuntu 9.10 for a while and wanted to see how it performs as a web server. The GUI interface is very slick and it is an ideal OS for beginners and advanced users alike. I suspect that many people will want to use it as the basis for their webservers.

I suggest creating a virtual machine with something like VirtualBox for this. It is much more flexible that way, and you can move it to a more powerful machine if you decide you need to.

Phase 1. Install required applications:

Open ‘Synaptic package manager’ from the System/Administration menu.
Click the ‘Search’ button.
Enter ‘Apache’ and press return.
You will see an item called ‘Apache2′ in the list of results. Click the box to the left to mark it as something you want to install.
Repeat this process for:
- Php5
- MySql
- Php5-mysql
Click the ‘Apply’ button on the toolbar.
This will start the installation process, during which you will be prompted to provide a password for the MySQL root account. Supply a strong password and make sure you do not forget it.
When everything has finished installing, I suggest rebooting. There are ways of starting the servers without rebooting, but a reboot works just as well and is much easier.
After you have logged back in to Ubuntu, open up Firefox and visit http://localhost.
You should see a page that says “It works!” which means that your webserver is now working.

Phase 2: Configure Virtual Hosts (So you can have more than one site on your server)

I have chosen to use the command line to do much of this phase rather than the GUI. For me, it is much easier to access protected system files from the shell rather than use a GUI. However, should you want to use a GUI, you will need to be running with root privileges. You can do this by entering: sudo nautilus in to a command line (and enter your password). However, the instructions below are for using the shell.

Any time you see the sudo command, that means that you will run the command as root (administrator). Linux is much more secure than Windows thanks to it being locked down by default with a standard set of permissions. It might seem like hassle to keep having to switch to be administrator but it makes things much safer. Deal with it!

Open a new Terminal from Applications / Accessories in the menus.
In the terminal, enter the following commands and review the results so you know what you are doing.
cd /etc/apache2/sites-enabled (this moves you to the folder where you can define different websites)
ls (lower case LS: this will show you a list of files)
sudo cp 000-default 001-mysite (this will copy the default config file so you can use it as the basis for your new site)
Enter your password when prompted (this should only happen once, but if you are prompted again – just re-enter it)
sudo gedit 001-mysite (this will use the gedit text editor to open and allow you to edit the new site config file)
Change the ServerAdmin row to use your email address. ie. ServerAdmin sam@mysite.net
Add a row below with the tag: ServerName followed by the name of your site. I will continue to use mysite.local throughout. ie. ServerName mysite.local
Edit the row DocumentRoot so that the new path is a sub-folder of www called 001-mysite. ie. /var/www/001-mysite
Edit the row so that the new path is also used. ie.
Save the file and closed GEdit.
We now need to let your computer know that it IS mysite.local. The easiest way to do this is to edit your hosts file. There is plenty of information online about what this file does.
From the terminal, type: sudo gedit /etc/hosts
Add a new row below 127.0.0.1 localhost that reads: 127.0.0.1 mysite.local (spaces are tabs).
Save the file and close GEdit.
Now we need to create the file structure for where the sites files will live.
Enter: sudo mkdir /var/www/001-mysite (this will create a new folder that will store your php/html files etc)
cd /var/www/001-mysite
sudo gedit index.php (this will open the text editor again and will allow you to create the index page for your site).
In the text editor, enter the following bold text (including all the punctuation):
Save the file and close.
sudo /etc/init.d/apache2 restart (this restarts the Apache service so that the new virtual host is available)
Now open firefox and visit your new site: http://mysite.local
You should see a whole load of information about your server, and more specifically about your PHP installation.
Thats it! Repeat this phase to add as many virtual hosts as you like.

You will now almost certainly want to secure your server. If you have got this far then you can at least get your site(s) up and running on your local machine. Ill cover securing your server in another post.

Inserting JSON data in to name-matched controls

Sam — Tue, 01 Sep 2009 12:08:10 +0000

Thought this might be handy… This loop will insert the values from a JSON object in to text boxes.

It loops around each key-pair and inserts the value if there is a text box with an ID that matches the key name.

This is the JSON data: (yourJson.js)

{"page":1,"total":1,"records":1,"paging":20,"rows":[
{"clipId":"14792","barcode":"02528","title":"My Project Sample","format":"DigiBeta","source":"1242",
"runningTime":"14 Mins","location":"Library","timeCode":"10:00:00","cUser":"Sam",
"cDate":"1 Jan 2007","mUser":"Sam","mDate":"1 Jan 2008"}]}

This is the form HTML:

<form>
	<table>
		<tr>
			<th>Clip Idth>
			<td><input id='clipId'>td>
		tr>
		<tr>
			<th>Barcodeth>
			<td><input id='barcode'>td>
		tr>
		<tr>
			<th>Titleth>
			<td><input id='title'>td>
		tr>
		<tr>
			<th>Formatth>
			<td><input id='format'>td>
		tr>
		<tr>
			<th>Sourceth>
			<td><input id='source'>td>
		tr>
		<tr>
			<th>Running Timeth>
			<td><input id='runningTime'>td>
		tr>
		<tr>
			<th>Locationth>
			<td><input id='location'>td>
		tr>
		<tr>
			<th>Time Codeth>
			<td><input id='timeCode'>td>
		tr>
	table>
form>

This is the jQuery:

	$.getJSON("/yourJson.js, function(myJson){
		$.each(myJson.rows, function(i,item) {
		    for (prop in item) {
				$('#' + prop).val(item[prop]);
		    }
		}); 
	});

Creating JSON data from MySQL in PHP

Sam — Mon, 24 Aug 2009 16:43:50 +0000

Here is a quick post that demonstrates how to create some neat JSON data from MySQL using PHP. The script is really simple, and has been working well for me over the last few weeks.

If anything is confusing, it might be worth checking out the previous post about creating XML data from PHP. It explains things in more detail and will help if you are not familiar…

//set up the php headers so that the page doesnt cache etc
header("Expires: Mon, 26 Jul 1997 05:00:00 GMT");
header("Last-Modified: " . gmdate( "D, d M Y H:i:s") . " GMT");
header("Cache-Control: no-cache, must-revalidate");
header("Pragma: no-cache");
header("Content-type: text/json");
 
$jsonArray->page = $page; 
$jsonArray->total = $totalPages;
$jsonArray->records = $recordCount; 
 
$i=0; 
while ($row = $db->fetch_array($countRows)) {
	$jsonArray->rows[$i]['id']=$row['userAccountId']; 
	$jsonArray->rows[$i]['userName']=$row['userName'];
	$jsonArray->rows[$i]['fullName']=$row['fullName'];
	$jsonArray->rows[$i]['accessCode']=$row['accessCode'];
	$jsonArray->rows[$i]['email']=$row['email'];
	$jsonArray->rows[$i]['addedBy']=$row['addedBy'];
	$jsonArray->rows[$i]['addedDate']=$row['addedDate'];
	$jsonArray->rows[$i]['modifiedBy']=$row['modifiedBy'];
	$jsonArray->rows[$i]['modifiedDate']=$row['modifiedDate'];
	$i++; 
} 
 
echo json_encode($jsonArray);

The above code will create the following style JSON output.

{"page":1,"total":1,"records":"3","rows":[
{"id":"1","userName":"demo","fullName":"Demo User","accessCode":"","email":"demo@myexample.com","addedBy":"1","addedDate":"2009-05-02 00:00:00","modifiedBy":"10","modifiedDate":"2009-06-04 00:00:00"},
{"id":"2","userName":"sam","fullName":"Sam Lasagne","accessCode":"","email":"sam@myexample.com","addedBy":"2","addedDate":"2009-05-02 00:00:00","modifiedBy":"2","modifiedDate":"2009-08-02 00:00:00"},
{"id":"10","userName":"adrian","fullName":"Adrian Spaghetti","accessCode":"","email":"adrian@myexample.com","addedBy":"1","addedDate":"2009-05-02 00:00:00","modifiedBy":"2","modifiedDate":"2009-09-01 00:00:00"}
]}

Using XML in your jQuery to populate input boxes

Sam — Tue, 11 Aug 2009 18:06:13 +0000

In a previous post I wrote about creating XML pages with MySQL data, using PHP. In this article, I’ll explain in simple terms how to parse that XML data and insert some of its values in to your page using jQuery.

I’m going to keep this very simple, as there seem to be very few examples of this on the net. It took me a while to work out a neat way of doing this, and I’m sure someone can improve on things. If so – please let me know in the comments.

First, lets set up an example. A very simple example.

Imagine an invoice. 4 boxes at the top of a grid, with a ‘Save’ button that will add the contents of those boxes to the list below. The fields are such as ‘Item code’, ‘Description’, ‘Quantity’ and ‘Unit Price’. The user enters the values and clicks the ‘Save’ button to add that row to the list and clear the boxes, ready for the next row to be added. For the purposes of what we are doing, we will also want a 5th box (which would usually be hidden) to store the ‘Invoice Detail Id’.

Something like this:




Code

Description

Quantity

Price



Hidden ID

Creating this application is all quite trivial for someone experienced in PHP. However, what would be nice is to be able to edit each row without having to re-load the page. Well we can do this quickly and easily using jQuery.

Before we start with the jQuery code, we need to have ready a PHP page that will return valid XML with the details of an invoice detail record. Please see my previous post for some help on how to do this.

This is an example output from the getDetailRow.php file.

 version="1.0" encoding="utf-8" ?>
>
  >
    >10>
    >CR282>
    >Software Support>
    >3>
    >40>
 >
>

Against each row, add a button labeled ‘Edit’, and attach some jQuery code that calls a function called editDetailRow(detailRowId) – as the grid is built, insert this code in to each button with the parameter of that row’s detail ID.

Assuming that there is only ever one result in the XML output, you could use the following code to insert the values in to your input boxes.

<script type="text/javascript">
function editDetailRow(detailId) {
    //first we need to load the XML data for that detail row
    //if the function is a success it will call the function called processDetail
    $.ajax({
       type: "GET",
       url: "getDetailRow.php?detailId=" + detailId,
       dataType: "xml",
       success: processDetail
     });
}
 
function processDetail(xml) {
    //this function gets the results from the xml file
    //and inserts them in to the boxes
    $(xml).find("invoiceDetail").each(function()   {
        $("#code").val($(this).find("code").text());
        $("#description").val($(this).find("description").text());
        $("#quantity").val($(this).find("quantity").text());
        $("#unitPrice").val($(this).find("unitPrice").text());
        $("#invoiceDetailId").val($(this).find("invoiceDetailId").text());
    });
}
script>

Yes, it really is that simple. You will need to add a reset button, and make sure you check for a value in the invoiceDetailId when you save (to either call an UPDATE or an INSERT) but thats it.

Hopefully you will find this useful and can use this as the basis for much more complicated XML and jQuery hookup’s.