gEt DoWn ToNiGhT

Full refresh in Mac Firefox

Sam — Fri, 11 May 2012 13:07:19 +0000

After a long time of clearing my cache out regularly for CSS / Javascript caching issues – I finally found out how to do a full refresh in Firefox on a Mac (clear cache etc).

CMD + SHIFT + R.

Example backup restore of MySQL databases

Sam — Sun, 06 May 2012 15:31:13 +0000

Quite often I need to backup a MySQL database, compress it, copy the data to another machine, then re-extract the data to MySQL.

Here is the process I go through for doing that.

mysqldump -p my_database_name > my_database_name_2012_05_06.sql

```
gzip my_database_name_2012_05_06.sql
```

scp my_database_name_2012_05_06.gz root@server2:/tmp

```
ssh root@server2
```
```
cd /tmp
```
```
gunzip my_database_name_2012_05_06.gz
```

mysql -p my_database_name < /tmp/my_database_name_2012_05_06.sql

Custom parameters and source on jQuery autocomplete

Sam — Sun, 29 Apr 2012 15:11:03 +0000

I have been working on converting our custom drop-down boxes within a php/jquery based app to use the jQuery UI ‘autocomplete’ tool, so that we can have keyboard-control over the selection choices (which our own code doesn’t have).

It took me quite a while to work out how to send custom parameters from other autocomplete boxes on the same page to the URL that gets our source JSON data, but after lots of playing around and searching on the web – I think I have finally cracked it – and I am recording it here in case I forget in the future!

2 parts. A PHP page for generating JSON data to use in the drop down, and a HTML/jQuery page that sets up the UI tool.

Here is the PHP page that I use to collect and return data in JSON format, based on some parameters given in a variety of ways:

$category       = get_input_request("category");
$description    = get_input_request("term");
$select_field   = get_input_request("selectField");
$select_value   = get_input_request("selectValue");
 
$order_by       = "description";
 
$xref_data      = get_xref_values($category, $order_by, $description, $select_field, $select_value);
 
$str            = get_autocomplete_string($xref_data);
 
header('Content-Type: application/json');
echo $str;

Here is the jQuery code:

$("#libraryLocation" ).autocomplete({
    matchContains: true,
    minLength: 0,
    source: function(request, response) {
        $.post("/dropDowns/autoXref.php", {
            category: 'location',
            selectField: 'format',
            selectValue: $('#libraryDepot').val(),
            term: $('#libraryLocation').val()
        }, function(result){
            response(result);
        })
    } 
}).click(function(){
    $("#libraryLocation" ).autocomplete('search', $(this).val());
});

Take a remote screenshot

Sam — Mon, 02 Apr 2012 21:03:40 +0000

If you need to take a screenshot of a machine and you only have terminal access – you can do the following:

DISPLAY=":0.0"; export DISPLAY; import -window root screengrab.png

Then you can download the file using SCP:

scp -P 22 sam@yourwebserver.com:screengrab.png ./screengrab.png

Sample Page for sending JSON data as string to web-service

Sam — Mon, 05 Dec 2011 16:56:04 +0000

Just realised it has been a year since I posted anything on here! Time really flies when you’re busy!

I remember when I was first starting off with jQuery, JSON and all that jazz that I struggled to find some simple examples of how to do things like send data to a web-service.

Here is a quick example of how to send some JSON data (as a single post field) to a web service, using jQuery’s post method.

Keep in mind that if you are posting to another domain you may need to use something called jsonP – but if you are just learning, this is a good starting point!



    
        JSON Web Service Demo
 
        
 
        
 
        <script type="text/javascript">
            $(document).ready(function() {
 
                //this code will be added to the onclick event
                $('#save_user_button').click(function() {
 
                    //add the values we need for the API to an object
                    var data = new Object;
 
                    //build the object
                    data.guid                   = $('#guid').val();
                    data.defaultallocation      = $('#projecttype').val();
                    data.email                  = $('#email').val();
                    data.username               = $('#username').val();
                    data.fullname               = $('#fullname').val();
 
                    //turn the object in to a JSON string
                    var myJson = JSON.stringify(data);
 
                    var url = "http://testurl/service/user";
 
                    //call the post
                    $.post(url, {jsonUser : myJson}, function(myResult) {
                        alert(myResult)
                    });
 
                });
 
            });
        script>
 
    
    
        
            
 
                
 
                
 
                
 
                
 
                
 
            
                    Project Type
                    
                

                    GUID
                    
                

                    User Name
                    
                

                    Full Name
                    
                

                    Email

Slow and Locked MySQL Queries

Sam — Fri, 17 Dec 2010 17:08:59 +0000

One of our clients’ servers recently saw a huge drop in speed. None of the hardware had been changed, and while the total number of users has been steadily increasing for the last few years, it has not seen any kind of spike that I thought might cause this type of problem.

In order to try to find the fault or problem, I had to look in many different places and discovered a lot about Linux and MySQL. I enjoy these fact finding missions, even though there are always a lot of dead ends.

I thought I would use this blog as a place to write down all the areas that are worth checking, in case anyone else finds themselves in this situation.

Things to check:

The Process List
MySQL Slow Query Log
Server free disk space
Server memory usage
Server load
Your queries
Your indexes

Our server was running on Linux Centos 3, and many of the commands below are Linux only. For executing the MySQL commands and queries I suggest getting something like WebYog’s excellent SQLYog, or using MySQL’s own MySQL Workbench.

Listing active MySQL queries

MySQL contains the ability to show a list of currently running queries. From within MySQL – execute the following query:

SHOW PROCESSLIST;

This will show a list that details which users, hosts, databases are being used to execute queries – how long they have been going for, and what state they are in. Look for queries that have been running for a long time, or have the state of ‘locked’. For more information see here.

Breaking down your queries

It is crucial that you enable slow query logging and it is quite simple to set up. Edit your my.cnf file, and make sure the following is present:

log-slow-queries = /var/log/mysql/mysql-slow.log
long_query_time = 3

(Adjust the log_query_time setting to suit the length of the query you are trying to detect)

Once this is in place, restart the MySQL server and start checking the new log file.

The MySQL command EXPLAIN allows you to see a breakdown of the steps taken by MySQL to get your query results, including information about joins and the sequence of queries called. You can use the information it outputs to help optimise your own queries.

The utility is easy to use. Simply prefix your SELECT statement with the word EXPLAIN. eg.

EXPLAIN
SELECT userName, firstName, lastName
FROM users
WHERE firstName LIKE 'da%';

top

This command shows you a list of applications and services running on the machine, along with some basic information about the resources they are using.

From here you can see the system load, free memory and the % of CPU for each process.

To quit and return to the shell, use CTRL+C.

df -h

This will show you how much free space is on your mounted drives, in human readable form.

[sam@webserver1 /]$ df -h
Filesystem            Size  Used Avail Use% Mounted on
/dev/md1              9.8G  3.9G  5.9G  40% /
none                 1014M     0 1014M   0% /dev/shm
/dev/md2               57G   31G   27G  54% /disk1

tail

This is a quick way of looking at the last few lines of a log file. The following will show you the last 10 lines of the mysql log.

tail /var/log/mysql.log

You can also specify that you want to view a specific number of lines by using the -n parameter. eg.

tail -n 50 /var/log/mysql.log

Eventually, after going through lots of checking and tests, we discovered that the server was simply at capacity. It hadn’t been upgraded for 8 years and the number of users had quadrupled during that time – with users from 2 other continents now logging on regularly. The database was on the same hardware as the web-server software and the system was just becoming overloaded.

We managed to speed things up considerably before we were forced to upgrade, mostly by examining the slow-query-log and adding indexes where appropriate. Also, many of our queries were refined and tidied up – so we were definitely not wasting our time running the examinations. It has made me realise that it is probably worth going back over old code every now and then to check for potential improvements – something we only tend to do if there is a client requirement, or an OS upgrade required.

We have now performed a complete upgrade to the latest hardware available, with 8GB of memory, more CPU cores than I suspect are really needed, the latest Centos 5 OS and all the most recent MySQL/Apache and PHP versions – which has made the system run incredibly quickly.

Using the database from inside a helper (CodeIgniter)

Sam — Thu, 17 Jun 2010 09:06:17 +0000

I have some custom functions that I store in a helper in CodeIgniter.

Some of those functions need to use the database, and to do so you have to explicitly tell the function to load the Code Igniter framework and database class.

Its pretty simple when you know how, but it took quite a while to work it out.

function get_user_full_name($userId) {
 
    //the database functions can not be called from within the helper
    //so we have to explicitly load the functions we need in to an object
    //that I will call ci. then we use that to access the regular stuff.
    $ci=& get_instance();
    $ci->load->database();
 
    //select the required fields from the database
    $ci->db->select('firstName, lastName');
 
    //tell the db class the criteria
    $ci->db->where('userId', $userId);
 
    //supply the table name and get the data
    $query = $ci->db->get('user');
 
    //ensure that there is something in the variable - in case of no match
    $fullName = "";
 
    foreach($query->result() as $row):
 
        //get the full name by concatinating the first and last names
        $fullName = $row->firstName . " " . $row->lastName;
 
    endforeach;
 
    // return the full name;
    return $fullName;
 
}

Using Eclipse as a PHP IDE in Ubuntu 9.10

Sam — Wed, 25 Nov 2009 17:05:40 +0000

This is actually much easier than any of the guides I found online make out.

Firstly, install Eclipse from the repositories. Then open it up, select your default workspace (if you havent already) and then:

From the Help menu, select “Install new software”.
Click the “Add” button and add the following:
- Name: Eclipse Updates (Galileo)
- Location: http://download.eclipse.org/releases/galileo/
Apply your changes and wait for the contents to download.
When all is ready, select your new source from the drop down and look for the “Programming languages” section.
Open it up and add the “PHP Development Tools (PDT) SDK Feature”.
Step through using Next and Finish etc, and after a few moments downloading and installing you now have a PHP ready IDE.

Loading JSON data – the EASY way!

Sam — Wed, 18 Nov 2009 12:22:04 +0000

A few weeks ago I wrote a blog post about using jQuery and JSON to automatically load data from a JSON file in to name-matched controls on an HTML page.

It is only now that I realise quite how powerful this is!

The great thing about it is that you do not have to edit your data-load function if you add a new field to the database. Because the jQuery function loops through all the fields in the JSON data, it will automatically insert the values in to the correct fields, providing those fields are named the same as the html form controls.

Here is the code again for those who dont want to check back.

$.getJSON("/yourJson.js, function(myJson){
	$.each(myJson.rows, function(i,item) {
		 for (prop in item) {
		 	 $('#' + prop).val(item[prop]);
		 }
	});
});

Installing MyODBC 3.51 on Windows 7 x64

Sam — Mon, 16 Nov 2009 11:11:44 +0000

In order to get some of our 32 bit applications working on Windows 7 64 bit edition, we need to provide support for the MySQL 3.51 ODBC connector.

This is actually quite simple to do, as long as you know what to look for. The ODBC manager that you are able to access from within Windows is not going to work for you, but it turns out that Microsoft provide a 32 bit manager that will.

To use this manager, run the following command:

c:\windows\syswow64\odbcad32.exe

Please note that it is not good enough to just type odbcad32.exe, as there is another file in the path with the same file name (great work MS!). However, once you have opened this application up, it works in the exact same way as if you were running XP.