<?xml version="1.0" encoding="UTF-8"?>
<rss version="2.0"
	xmlns:content="http://purl.org/rss/1.0/modules/content/"
	xmlns:wfw="http://wellformedweb.org/CommentAPI/"
	xmlns:dc="http://purl.org/dc/elements/1.1/"
	xmlns:atom="http://www.w3.org/2005/Atom"
	xmlns:sy="http://purl.org/rss/1.0/modules/syndication/"
	xmlns:slash="http://purl.org/rss/1.0/modules/slash/"
	>

<channel>
	<title>HBY Consultancy &#187; Blog</title>
	<atom:link href="http://www.hbyconsultancy.com/blog/feed" rel="self" type="application/rss+xml" />
	<link>http://www.hbyconsultancy.com</link>
	<description>IT Consultant, PHP Expert, e-Government Specialist and Energy Engineer</description>
	<lastBuildDate>Mon, 09 Aug 2010 07:03:10 +0000</lastBuildDate>
	<language>en</language>
	<sy:updatePeriod>hourly</sy:updatePeriod>
	<sy:updateFrequency>1</sy:updateFrequency>
			<item>
		<title>Shared OCFS2 partition on Ubuntu Server 10.04 x64</title>
		<link>http://www.hbyconsultancy.com/blog/shared-ocfs2-partition-on-ubuntu-server-10-04-x64.html</link>
		<comments>http://www.hbyconsultancy.com/blog/shared-ocfs2-partition-on-ubuntu-server-10-04-x64.html#comments</comments>
		<pubDate>Sat, 07 Aug 2010 09:08:11 +0000</pubDate>
		<dc:creator>hatem</dc:creator>
				<category><![CDATA[Blog]]></category>
		<category><![CDATA[cluster]]></category>
		<category><![CDATA[GFS2]]></category>
		<category><![CDATA[HBA Card]]></category>
		<category><![CDATA[NFS]]></category>
		<category><![CDATA[OCFS2]]></category>
		<category><![CDATA[Oracle]]></category>
		<category><![CDATA[SAN Storage]]></category>
		<category><![CDATA[Shared Filesystem]]></category>
		<category><![CDATA[Ubuntu]]></category>

		<guid isPermaLink="false">http://www.hbyconsultancy.com/?p=494</guid>
		<description><![CDATA[One of the applications that I&#8217;m working on uses archived documents, there is no NoSQL here, just plain tiff files with indexes in Oracle database. Everything related to document access, permissions, conversions, watermarking, security, encryption &#8230; is managed by the application itself. So I had to keep my cluster permanently connected to a SAN storage [...]


Related posts:<ol><li><a href='http://www.hbyconsultancy.com/blog/two-nodes-load-balance-and-failover-with-keepalived-and-ubuntu-server-10-04-x64.html' rel='bookmark' title='Permanent Link: Two nodes Load balance and Failover with keepalived and Ubuntu Server 10.04 x64'>Two nodes Load balance and Failover with keepalived and Ubuntu Server 10.04 x64</a></li>
<li><a href='http://www.hbyconsultancy.com/blog/create-a-master-master-mysql-replication-ubuntu-server-10-04-x64.html' rel='bookmark' title='Permanent Link: Create a Master Master MySQL replication &#8211; Ubuntu Server 10.04 x64'>Create a Master Master MySQL replication &#8211; Ubuntu Server 10.04 x64</a></li>
</ol>]]></description>
			<content:encoded><![CDATA[<p>One of the applications that I&#8217;m working on uses archived documents, there is no NoSQL here, just plain tiff files with indexes in Oracle database. Everything related to document access, permissions, conversions, watermarking, security, encryption &#8230; is managed by the application itself. So I had to keep <a href="http://www.hbyconsultancy.com/blog/two-nodes-load-balance-and-failover-with-keepalived-and-ubuntu-server-10-04-x64.html">my cluster</a> permanently connected to a SAN storage via HBA fiber cards.</p>
<p>In the beginning I opted for NFS as shared file system, then decided to go for <a href="http://oss.oracle.com/projects/ocfs2/">OCFS2</a> which is open source, maintained by Oracle, and available under GPLv2.</p>
<p>Some Pros/Cons of NFS, GFS2 and OCFS2 (from <a href="http://www.gpaterno.com/publications/2010/dublin_ossbarcamp_2010_fs_comparison.pdf">Dublin OSS barcamp</a>) :</p>
<p><strong>NFS</strong></p>
<ul>
<li>Pro: standard, cross-platform, easy to implement</li>
<li>Con: Poor performance, single point of failure (single locking manager, even in HA)</li>
</ul>
<p><strong>GFS2</strong></p>
<ul>
<li>Pro: Very responsive on large data files, works on physical and virtual, quota and SE-Linux support, faster than EXT3 when I/O operations are on the same node</li>
<li>Con: Only supported with Red Hat, Performance issues on accessing small files on several subdirectory on different nodes</li>
</ul>
<p><strong>OCFS2</strong></p>
<ul>
<li>Pro: Very fast with large and small data files on different node with two types of performance models (mail, data file). Works on a physical and virtual.</li>
<li>Con: Supported only through contract with Oracle or SLES, no quota support, no on-line resize</li>
</ul>
<p>First we need to install OCFS2 tools :</p>
<p><code>sudo apt-get install ocfs2-tools</code></p>
<p>There is another package ocfs2console that you want to install to configure the cluster via GUI, but since I&#8217;m using ubuntu server I&#8217;m skipping this to configure my cluster manually.</p>
<p style="text-align: center;"><a href="http://www.hbyconsultancy.com/wp-content/uploads/2010/08/CFS2Console.png"><img class="size-medium wp-image-496 aligncenter" title="CFS2Console" src="http://www.hbyconsultancy.com/wp-content/uploads/2010/08/CFS2Console-300x192.png" alt="" width="300" height="192" /></a></p>
<p>Create on every node attached to storage /etc/ocfs2/cluster.conf</p>
<p><code>sudo vi /etc/ocfs2/cluster.conf</code></p>
<p>With the content below, only replace node1 and node1 with their respective names and IP for each node :</p>
<p><code>node:<br />
name = node1<br />
cluster = ocfs2<br />
number = 0<br />
ip_address = 10.10.0.0<br />
ip_port = 7777<br />
node:<br />
name = node2<br />
cluster = ocfs2<br />
number = 1<br />
ip_address = 10.10.0.1<br />
ip_port = 7777<br />
cluster:<br />
name = ocfs2<br />
node_count = 2</code></p>
<p>Now you reconfigure ocfs2-tools with the default values :</p>
<p><code>sudo dpkg-reconfigure ocfs2-tools</code></p>
<p>then restart services :</p>
<p><code>sudo /etc/init.d/o2cb restart<br />
sudo /etc/init.d/ocfs2 restart</code></p>
<p>If your fiber card connected to your host/storage, and virtual disks created and presented you should run fdisk to see it :</p>
<p><code>$ sudo fdisk -l<br />
Disk /dev/sda: 1073.7 GB, 1073741824000 bytes<br />
255 heads, 63 sectors/track, 130541 cylinders<br />
Units = cylinders of 16065 * 512 = 8225280 bytes<br />
Sector size (logical/physical): 512 bytes / 512 bytes<br />
I/O size (minimum/optimal): 512 bytes / 512 bytes<br />
Disk identifier: 0x02020202</code></p>
<p>the result have been truncated to show only one virtual disk, and you might see multiple /dev/sda, /dev/sdb, /dev/sdc&#8230; according to your configuration in addition to your local hard disks. What I have done is creating a 1TB partition that I will share between my two nodes :</p>
<p><code>$ sudo fdisk /dev/sda</code></p>
<p>In fdisk menu choose &#8220;n&#8221; for new partition, and choose your partition size according to your requirements. Then use &#8220;w&#8221; to write changes and exit.</p>
<p>Finally we create a ocfs2 partition table :</p>
<p><code>$ mkfs.ocfs2 /dev/sda</code></p>
<p>mount your partition :</p>
<p><code>$ sudo mkdir /archives<br />
$sudo mount -t ocfs2 /dev/sda /archives</code></p>
<p>or you can add it to fstab to mount automatically on boot :</p>
<p><code>$ /dev/sda /archives ocfs2 _netdev 0 0</code></p>
<p>The _netdev option is used here to prevent the system from attempting to mount these file systems until the network has been enabled on the system.</p>
<p>You want to test your new partition, and you will notice that every file/folder created on node1 is automatically available on node2, and vise-versa.</p>
<p>Enjoy !</p>


<p>Related posts:<ol><li><a href='http://www.hbyconsultancy.com/blog/two-nodes-load-balance-and-failover-with-keepalived-and-ubuntu-server-10-04-x64.html' rel='bookmark' title='Permanent Link: Two nodes Load balance and Failover with keepalived and Ubuntu Server 10.04 x64'>Two nodes Load balance and Failover with keepalived and Ubuntu Server 10.04 x64</a></li>
<li><a href='http://www.hbyconsultancy.com/blog/create-a-master-master-mysql-replication-ubuntu-server-10-04-x64.html' rel='bookmark' title='Permanent Link: Create a Master Master MySQL replication &#8211; Ubuntu Server 10.04 x64'>Create a Master Master MySQL replication &#8211; Ubuntu Server 10.04 x64</a></li>
</ol></p>]]></content:encoded>
			<wfw:commentRss>http://www.hbyconsultancy.com/blog/shared-ocfs2-partition-on-ubuntu-server-10-04-x64.html/feed</wfw:commentRss>
		<slash:comments>0</slash:comments>
		</item>
		<item>
		<title>Two nodes Load balance and Failover with keepalived and Ubuntu Server 10.04 x64</title>
		<link>http://www.hbyconsultancy.com/blog/two-nodes-load-balance-and-failover-with-keepalived-and-ubuntu-server-10-04-x64.html</link>
		<comments>http://www.hbyconsultancy.com/blog/two-nodes-load-balance-and-failover-with-keepalived-and-ubuntu-server-10-04-x64.html#comments</comments>
		<pubDate>Sun, 01 Aug 2010 10:23:43 +0000</pubDate>
		<dc:creator>hatem</dc:creator>
				<category><![CDATA[Blog]]></category>
		<category><![CDATA[Apache]]></category>
		<category><![CDATA[cluster]]></category>
		<category><![CDATA[Failover]]></category>
		<category><![CDATA[IPVS]]></category>
		<category><![CDATA[Keepalived]]></category>
		<category><![CDATA[Load balance]]></category>
		<category><![CDATA[Ubuntu]]></category>

		<guid isPermaLink="false">http://www.hbyconsultancy.com/?p=477</guid>
		<description><![CDATA[In an ideal system architecture using load balancers in separate nodes is preferred, however it&#8217;s also possible to have your load balancers in the same nodes with your applications. I have used in this architecture the same hardware as the previous Master/Master MySQL cluster, including Ubuntu server 10.04 x64, Apache2 as web server, two nodes [...]


Related posts:<ol><li><a href='http://www.hbyconsultancy.com/blog/shared-ocfs2-partition-on-ubuntu-server-10-04-x64.html' rel='bookmark' title='Permanent Link: Shared OCFS2 partition on Ubuntu Server 10.04 x64'>Shared OCFS2 partition on Ubuntu Server 10.04 x64</a></li>
<li><a href='http://www.hbyconsultancy.com/blog/create-a-master-master-mysql-replication-ubuntu-server-10-04-x64.html' rel='bookmark' title='Permanent Link: Create a Master Master MySQL replication &#8211; Ubuntu Server 10.04 x64'>Create a Master Master MySQL replication &#8211; Ubuntu Server 10.04 x64</a></li>
</ol>]]></description>
			<content:encoded><![CDATA[<p>In an ideal system architecture using load balancers in separate nodes is preferred, however it&#8217;s also possible to have your load balancers in the same nodes with your applications. I have used in this architecture the same hardware as the previous <a href="http://www.hbyconsultancy.com/blog/create-a-master-master-mysql-replication-ubuntu-server-10-04-x64.html">Master/Master MySQL cluster</a>, including Ubuntu server 10.04 x64, Apache2 as web server, two nodes HP DL380G6 with 3 hard disks 15K in RAID5 and connected to a SAN storage via Fiber. For load balancing and failover I used keepalived and LVS, and you can use heartbeat to get your cluster running.</p>
<p>First you will need to set at least two IPs (10.10.0.1 and 10.10.0.2) for your servers, and one virtual (10.10.0.3) shared between the two servers, you will have for the first interface :</p>
<p><code># The primary network interface<br />
auto <strong>eth0</strong><br />
iface eth0 inet static<br />
address <strong>10.10.0.1</strong><br />
netmask 255.255.255.0<br />
network 10.10.0.0<br />
broadcast 10.10.0.255<br />
gateway 10.10.0.250<br />
<strong>auto eth0:0</strong><br />
iface eth0:0 inet static<br />
address <strong>10.10.0.3</strong><br />
netmask 255.255.255.0<br />
network 10.10.0.0<br />
broadcast 10.10.0.255</code></p>
<p>and the second interface :</p>
<p><code># The primary network interface<br />
auto <strong>eth0</strong><br />
iface eth0 inet static<br />
address <strong>10.10.0.2</strong><br />
netmask 255.255.255.0<br />
network 10.10.0.0<br />
broadcast 10.10.0.255<br />
gateway 10.10.0.250<br />
<strong>auto eth0:0</strong><br />
iface eth0:0 inet static<br />
address <strong>10.10.0.3</strong><br />
netmask 255.255.255.0<br />
network 10.10.0.0<br />
broadcast 10.10.0.255</code></p>
<p>Then we can start by installing keepalived (v1.1.17 is available in Ubuntu repositories)</p>
<p><code>sudo apt-get install keepalived</code></p>
<p>You will have to create two configuration files for the first node 10.10.0.1 (Master) and second node 10.10.0.2 (Backup). So we add in the master node :</p>
<p><code>usr01@server01:~$ sudo nano /etc/keepalived/keepalived.conf<br />
# Keepalived Configuration File<br />
vrrp_instance VI_1 {<br />
state <strong>MASTER</strong><br />
interface eth0<br />
virtual_router_id 10<br />
priority <strong>200</strong><br />
virtual_ipaddress {<br />
10.10.0.3/24<br />
}<br />
notify_master "/etc/keepalived/notify.sh del 10.10.0.3"<br />
notify_backup "/etc/keepalived/notify.sh add 10.10.0.3"<br />
notify_fault "/etc/keepalived/notify.sh add 10.10.0.3"<br />
}<br />
virtual_server 10.10.0.3 80 {<br />
delay_loop 30<br />
lb_algo <strong>rr</strong><br />
lb_kind <strong>DR</strong><br />
persistence_timeout 50<br />
protocol TCP<br />
real_server 10.10.0.1 80 {<br />
weight 100<br />
HTTP_GET {<br />
url {<br />
path /index.php<br />
digest d41d8cd98f00b204e9800998ecf8427e<br />
}<br />
connect_timeout 3<br />
nb_get_retry 3<br />
delay_before_retry 2<br />
}<br />
}<br />
real_server 10.10.0.2 80 {<br />
weight 100<br />
HTTP_GET {<br />
url {<br />
path /index.php<br />
digest d41d8cd98f00b204e9800998ecf8427e<br />
}<br />
connect_timeout 3<br />
nb_get_retry 3<br />
delay_before_retry 2<br />
}<br />
}<br />
}<br />
</code></p>
<p>And in the backup node :</p>
<p><code>usr01@server02:~$ cat /etc/keepalived/keepalived.conf<br />
# Keepalived Configuration File<br />
vrrp_instance VI_1 {<br />
state <strong>BACKUP</strong><br />
interface eth0<br />
virtual_router_id 10<br />
priority <strong>100</strong><br />
virtual_ipaddress {<br />
10.10.0.3/24<br />
}<br />
notify_master "/etc/keepalived/notify.sh del 10.10.0.3"<br />
notify_backup "/etc/keepalived/notify.sh add 10.10.0.3"<br />
notify_fault "/etc/keepalived/notify.sh add 10.10.0.3"<br />
}<br />
virtual_server 10.10.0.3 80 {<br />
delay_loop 30<br />
lb_algo <strong>rr</strong><br />
lb_kind <strong>DR</strong><br />
persistence_timeout 50<br />
protocol TCP<br />
real_server 10.10.0.1 80 {<br />
weight 100<br />
HTTP_GET {<br />
url {<br />
path /check.txt<br />
digest d41d8cd98f00b204e9800998ecf8427e<br />
}<br />
connect_timeout 3<br />
nb_get_retry 3<br />
delay_before_retry 2<br />
}<br />
}<br />
real_server 10.10.0.2 80 {<br />
weight 100<br />
HTTP_GET {<br />
url {<br />
path /check.txt<br />
digest d41d8cd98f00b204e9800998ecf8427e<br />
}<br />
connect_timeout 3<br />
nb_get_retry 3<br />
delay_before_retry 2<br />
}<br />
}<br />
}</code></p>
<p>The hash is created using, notice that you can add exception so apache don&#8217;t log check.txt requests.</p>
<p><code>usr01@server01:~$ genhash -s 10.10.0.1 -p 80 -u /check.txt<br />
MD5SUM = d41d8cd98f00b204e9800998ecf8427e<br />
usr01@server01:~$ genhash -s 10.10.0.2 -p 80 -u /check.txt<br />
MD5SUM = d41d8cd98f00b204e9800998ecf8427e</code></p>
<p>Also in both nodes we have to add a small utility to notify :</p>
<p><code><br />
#!/bin/bash<br />
VIP="$2"<br />
case "$1" in<br />
add)<br />
/sbin/iptables -A PREROUTING -t nat -d $VIP -p tcp -j REDIRECT<br />
;;<br />
del)<br />
/sbin/iptables -D PREROUTING -t nat -d $VIP -p tcp -j REDIRECT<br />
;;<br />
*)<br />
echo "Usage: $0 {add|del} ipaddress"<br />
exit 1<br />
esac<br />
exit 0</code></p>
<p>Launch keepalived on the two nodes :</p>
<p><code>sudo /etc/init.d/keepalived start</code></p>
<p>Now we need to enable ip_forward on the two nodes permanently</p>
<p><code>net.ipv4.ip_forward = 1</code></p>
<p>restart network on the two nodes</p>
<p><code>sudo /etc/init.d/networking  restart</code></p>
<p>And we can check that load balancing is working correctly on Master :</p>
<p><code>usr01@server02:~$ sudo ipvsadm -L -n<br />
[sudo] password for usr01:<br />
IP Virtual Server version 1.2.1 (size=4096)<br />
Prot LocalAddress:Port Scheduler Flags<br />
-&gt; RemoteAddress:Port           Forward Weight ActiveConn InActConn<br />
TCP  10.10.0.3:80 <strong>rr persistent 50</strong><br />
-&gt; 10.10.0.1:80 <strong> Local</strong> 100    0          0<br />
-&gt; 10.10.0.2:80 <strong> Route</strong> 100    0          0 </code></p>
<p>Also on Backup server</p>
<p><code>usr01@server02:~$ sudo ipvsadm -L -n<br />
[sudo] password for usr01:<br />
IP Virtual Server version 1.2.1 (size=4096)<br />
Prot LocalAddress:Port Scheduler Flags<br />
-&gt; RemoteAddress:Port           Forward Weight ActiveConn InActConn<br />
TCP  10.10.0.3:80 <strong>rr persistent 50</strong><br />
-&gt; 10.10.0.1:80 <strong> Route</strong> 100    0          0<br />
-&gt; 10.10.0.2:80 <strong> Local</strong> 100    0          0 </code></p>
<p>We are almost done, we only need to add a preroute rule on the backup node manually to get started :</p>
<p><code>usr01@server02$ iptables -A PREROUTING -t nat -d 10.10.0.3 -p tcp -j REDIRECT<br />
usr01@server02$ iptables -t nat --list<br />
Chain PREROUTING (policy ACCEPT)<br />
target     prot opt source               destination<br />
<strong> REDIRECT   tcp  --  anywhere             10.10.0.3 </strong><br />
Chain POSTROUTING (policy ACCEPT)<br />
target     prot opt source               destination<br />
Chain OUTPUT (policy ACCEPT)<br />
target     prot opt source               destination</code></p>
<p>That&#8217;s all.</p>
<p>Now you can connect to http://10.10.0.3 and you can notice load distributed between two nodes internally. In case one of the nodes fail, it will takes few seconds until the backup server notice the failure and update its iptables prerouting rule. When apache service goes down, you will notice that request on port 80 will be automatically redirected to second node.</p>
<p>As I have mentioned in the beginning, failover control cannot goes without downtime in such architecture, but it still great to distribute load if you are limited in hardware.</p>
<p>Finally, it will be much easier (even faster) to load balance using Round Robin DNS from active directory for example, if you can manage to monitor failed service or node, however this architecture remain better on failover even with a short downtime.</p>


<p>Related posts:<ol><li><a href='http://www.hbyconsultancy.com/blog/shared-ocfs2-partition-on-ubuntu-server-10-04-x64.html' rel='bookmark' title='Permanent Link: Shared OCFS2 partition on Ubuntu Server 10.04 x64'>Shared OCFS2 partition on Ubuntu Server 10.04 x64</a></li>
<li><a href='http://www.hbyconsultancy.com/blog/create-a-master-master-mysql-replication-ubuntu-server-10-04-x64.html' rel='bookmark' title='Permanent Link: Create a Master Master MySQL replication &#8211; Ubuntu Server 10.04 x64'>Create a Master Master MySQL replication &#8211; Ubuntu Server 10.04 x64</a></li>
</ol></p>]]></content:encoded>
			<wfw:commentRss>http://www.hbyconsultancy.com/blog/two-nodes-load-balance-and-failover-with-keepalived-and-ubuntu-server-10-04-x64.html/feed</wfw:commentRss>
		<slash:comments>1</slash:comments>
		</item>
		<item>
		<title>Why ECM &amp; e-Archiving Solutions Should Adopt NoSQL</title>
		<link>http://www.hbyconsultancy.com/blog/ecm-e-archiving-solutions-should-adopt-nosql.html</link>
		<comments>http://www.hbyconsultancy.com/blog/ecm-e-archiving-solutions-should-adopt-nosql.html#comments</comments>
		<pubDate>Sat, 31 Jul 2010 22:18:17 +0000</pubDate>
		<dc:creator>hatem</dc:creator>
				<category><![CDATA[Blog]]></category>
		<category><![CDATA[DMS]]></category>
		<category><![CDATA[e-Archiving]]></category>
		<category><![CDATA[e-Government]]></category>
		<category><![CDATA[ECM]]></category>
		<category><![CDATA[NoSQL]]></category>

		<guid isPermaLink="false">http://www.hbyconsultancy.com/?p=472</guid>
		<description><![CDATA[I still can&#8217;t get why most ECM, DMS, e-Archiving solutions are in love with RDBMS, whatever the technology they adopt : MSQL, DB2, Oracle, &#8230; it&#8217;s not only about additional licenses (and cost) for the entire solution, but also about features and performance. The key feature in DMS, ECM, or e-Archiving solutions is the &#8220;Document&#8221; [...]


Related posts:<ol><li><a href='http://www.hbyconsultancy.com/blog/say-yes-to-open-source.html' rel='bookmark' title='Permanent Link: Say YES to Open Source'>Say YES to Open Source</a></li>
<li><a href='http://www.hbyconsultancy.com/blog/yefi-e-government-interoperability-framework.html' rel='bookmark' title='Permanent Link: YEFI e-Government Interoperability Framework'>YEFI e-Government Interoperability Framework</a></li>
</ol>]]></description>
			<content:encoded><![CDATA[<p>I still can&#8217;t get why most ECM, DMS, e-Archiving solutions are in love with RDBMS, whatever the technology they adopt : MSQL, DB2, Oracle, &#8230; it&#8217;s not only about additional licenses (and cost) for the entire solution, but also about features and performance.</p>
<p>The key feature in DMS, ECM, or e-Archiving solutions is the &#8220;Document&#8221; itself. It&#8217;s all about creating documents and retrieving them, everything else is built on this feature : security, workflow, search and index, backup, restore, encryption &#8230; etc.</p>
<p>The NoSQL movement have the answer for most ECM implementations since it&#8217;s mainly about key/value data store, where you can mainly store documents (of data) and retrieve them.</p>
<p>In a NoSQL environment there is no lock, so your ECM is highly available, scalable, and replicated across your system. What else are we looking for ? Oh yes consistency !</p>
<p>The &#8220;eventually consistent&#8221; is probably the only negative point for adopting NoSQL against the &#8220;enforced consistency&#8221; of RDBMS. In the same time you cannot easily distribute RDBMS and scale it horizontally, which is more fun to do in NoSQL.</p>
<p><a href="http://www.hbyconsultancy.com/wp-content/uploads/2010/07/CAP-theorem.png"><img class="alignnone size-full wp-image-473" title="CAP theorem" src="http://www.hbyconsultancy.com/wp-content/uploads/2010/07/CAP-theorem.png" alt="" width="500" height="375" /></a></p>
<p>DMS of the future will need to adopt NoSQL  :</p>
<ul class="secondList">
<li>because new systems are built for Internet : highly available documents is required feature, imagine a place where everybody can really write simultaneously &#8211; no locks-in.</li>
<li>If you request a specific document, you will get it and there is no difference here with RDBMS, noSQL is even more performing.</li>
<li>The &#8220;eventually consistent&#8221; will not really change anything, when you need a global view of the data (stats for example) you will get it &#8220;consistent&#8221;.</li>
<li>Backup of documents could be done easily &#8211; and you will fall in love with replication</li>
<li>Sharding is your friend for large distributed database of documents</li>
</ul>
<p>I won&#8217;t need to mention successful case studies, because most cases are trying to replace RDBMS for relational databases, while DMS is mainly about documents. Using NoSQL database coupled with an RDBMS could also be a great alternative to provide additional features for documents management.</p>
<p>Lots of great NoSQL implementation already available such as CouchDB, MongoDB, BerkleyDB, Neo4j, Memcachedb, Tokyo Cabinet, BigTable, Redis, &#8230; etc. and not a single DMS or e-Archiving initiative to adopt NoSQL ?</p>
<p>IBM Filenet, OpenText, LaserFiche, and all other companies working in the ECM, DMS, e-Archiving business should be worried, the future will be Not Only for them.</p>
<p><strong>Update : </strong> as Alex mentioned on <a href="http://nosql.mypopescu.com/post/893770867/nosql-databases-and-cms-or-ecm-or-dms">myNoSQL Magazine</a>, and to avoid confusion, I don&#8217;t support NoSQL for content management in general, however ECM is mentioned here as it usually support document management, assets management, &#8230; etc.</p>


<p>Related posts:<ol><li><a href='http://www.hbyconsultancy.com/blog/say-yes-to-open-source.html' rel='bookmark' title='Permanent Link: Say YES to Open Source'>Say YES to Open Source</a></li>
<li><a href='http://www.hbyconsultancy.com/blog/yefi-e-government-interoperability-framework.html' rel='bookmark' title='Permanent Link: YEFI e-Government Interoperability Framework'>YEFI e-Government Interoperability Framework</a></li>
</ol></p>]]></content:encoded>
			<wfw:commentRss>http://www.hbyconsultancy.com/blog/ecm-e-archiving-solutions-should-adopt-nosql.html/feed</wfw:commentRss>
		<slash:comments>0</slash:comments>
		</item>
		<item>
		<title>Say YES to Open Source</title>
		<link>http://www.hbyconsultancy.com/blog/say-yes-to-open-source.html</link>
		<comments>http://www.hbyconsultancy.com/blog/say-yes-to-open-source.html#comments</comments>
		<pubDate>Mon, 26 Jul 2010 22:05:06 +0000</pubDate>
		<dc:creator>hatem</dc:creator>
				<category><![CDATA[Blog]]></category>
		<category><![CDATA[BPM]]></category>
		<category><![CDATA[e-Business]]></category>
		<category><![CDATA[e-Government]]></category>
		<category><![CDATA[Open Source]]></category>
		<category><![CDATA[University]]></category>
		<category><![CDATA[Yesser]]></category>

		<guid isPermaLink="false">http://www.hbyconsultancy.com/?p=469</guid>
		<description><![CDATA[Today I was reading on Okaz newspaper a news about a contractor who abused of their confidence and programmed a password change after leaving his position at King Abdul Aziz University. Things that creates troubles in administrative transactions in the entire university. Without entering into further details, how this could happen, and why &#8230; It&#8217;s [...]


Related posts:<ol><li><a href='http://www.hbyconsultancy.com/blog/open-source-closed-source-are-not-ennemie.html' rel='bookmark' title='Permanent Link: Open Source, Closed Source, We are not enemies'>Open Source, Closed Source, We are not enemies</a></li>
<li><a href='http://www.hbyconsultancy.com/blog/open-source-evangelism.html' rel='bookmark' title='Permanent Link: Open Source Evangelism'>Open Source Evangelism</a></li>
<li><a href='http://www.hbyconsultancy.com/blog/ecm-e-archiving-solutions-should-adopt-nosql.html' rel='bookmark' title='Permanent Link: Why ECM &#038; e-Archiving Solutions Should Adopt NoSQL'>Why ECM &#038; e-Archiving Solutions Should Adopt NoSQL</a></li>
</ol>]]></description>
			<content:encoded><![CDATA[<p>Today I was reading <a href="http://www.okaz.com.sa/new/Issues/20100726/Con20100726363732.htm">on Okaz newspaper</a> a news about a contractor who abused of their confidence and programmed a password change after leaving his position at King Abdul Aziz University. Things that creates troubles in administrative transactions in the entire university.</p>
<p><a href="http://www.hbyconsultancy.com/wp-content/uploads/2010/07/2010-07-26-23.48.26.jpg"><img class="alignnone size-full wp-image-470" title="2010-07-26 23.48.26" src="http://www.hbyconsultancy.com/wp-content/uploads/2010/07/2010-07-26-23.48.26.jpg" alt="" width="400" height="300" /></a></p>
<p>Without entering into further details, how this could happen, and why &#8230; It&#8217;s true that something wrong happened and they assumed the consequences. I&#8217;m not talking about the small amount of money they paid to get back the password, but about the scam behind it and the name of the university in question.</p>
<p>I wanted to profit &#8211; as usual &#8211; and focus again on the strategic decisions of considering Open source as alternative in critical mission projects. Whatever if you are talking about ERP, CRM, BPM, DMS, &#8230; the open source alternative is always available and costless compared to proprietary solutions. Best of all, open source is a guarantee to have full control on the business and get lock-free solution.</p>
<p>Imagine if this project was made in an open source ecosystem, the university will never get locks anywhere, and their system might be updated easily and quickly &#8211; even if something wrong happened.</p>
<p>Most important things to consider to avoid locks :</p>
<ul>
<li>Adopt Open Source technologies</li>
<li>Document everything : Business, communications, code and databases</li>
<li>Adopt a backup/restore strategy</li>
<li>Adopt a disaster recovery strategy</li>
<li>Share the knowledge in a development team, and avoid using only one developer in a large project to minimize cost. Two or more developers will get work done faster and safer.</li>
<li>Separate critical mission projects, from new alpha/beta projects. Use SOA as much as possible for integration.</li>
<li>Do not go e-Business if you are not ready for it and always provide alternative in case of failure. And consider seriously that the project will fail.</li>
<li>finally -in Saudi Arabia specifically- ask for support, <a href="http://www.yesser.gov.sa">Yesser</a> program is doing great work and their consultants might really help to secure your e-Business.</li>
</ul>
<p>You still don&#8217;t trust open source ?</p>


<p>Related posts:<ol><li><a href='http://www.hbyconsultancy.com/blog/open-source-closed-source-are-not-ennemie.html' rel='bookmark' title='Permanent Link: Open Source, Closed Source, We are not enemies'>Open Source, Closed Source, We are not enemies</a></li>
<li><a href='http://www.hbyconsultancy.com/blog/open-source-evangelism.html' rel='bookmark' title='Permanent Link: Open Source Evangelism'>Open Source Evangelism</a></li>
<li><a href='http://www.hbyconsultancy.com/blog/ecm-e-archiving-solutions-should-adopt-nosql.html' rel='bookmark' title='Permanent Link: Why ECM &#038; e-Archiving Solutions Should Adopt NoSQL'>Why ECM &#038; e-Archiving Solutions Should Adopt NoSQL</a></li>
</ol></p>]]></content:encoded>
			<wfw:commentRss>http://www.hbyconsultancy.com/blog/say-yes-to-open-source.html/feed</wfw:commentRss>
		<slash:comments>0</slash:comments>
		</item>
		<item>
		<title>Create a Master Master MySQL replication &#8211; Ubuntu Server 10.04 x64</title>
		<link>http://www.hbyconsultancy.com/blog/create-a-master-master-mysql-replication-ubuntu-server-10-04-x64.html</link>
		<comments>http://www.hbyconsultancy.com/blog/create-a-master-master-mysql-replication-ubuntu-server-10-04-x64.html#comments</comments>
		<pubDate>Mon, 19 Jul 2010 12:41:23 +0000</pubDate>
		<dc:creator>hatem</dc:creator>
				<category><![CDATA[Blog]]></category>
		<category><![CDATA[high availability]]></category>
		<category><![CDATA[master master replication]]></category>
		<category><![CDATA[mysql]]></category>
		<category><![CDATA[mysql cluster]]></category>
		<category><![CDATA[system architecture]]></category>

		<guid isPermaLink="false">http://www.hbyconsultancy.com/?p=446</guid>
		<description><![CDATA[There are lots of howtos about creating a MySQL cluster, and this is another one that might be useful in some special cases. In this tutorial I will create a Two nodes Master/Master MySQL replication on Ubuntu server 10.04. I used in my cluster two nodes HP DL380G6 with 3 hard disks 15K in RAID5 [...]


Related posts:<ol><li><a href='http://www.hbyconsultancy.com/blog/shared-ocfs2-partition-on-ubuntu-server-10-04-x64.html' rel='bookmark' title='Permanent Link: Shared OCFS2 partition on Ubuntu Server 10.04 x64'>Shared OCFS2 partition on Ubuntu Server 10.04 x64</a></li>
<li><a href='http://www.hbyconsultancy.com/blog/two-nodes-load-balance-and-failover-with-keepalived-and-ubuntu-server-10-04-x64.html' rel='bookmark' title='Permanent Link: Two nodes Load balance and Failover with keepalived and Ubuntu Server 10.04 x64'>Two nodes Load balance and Failover with keepalived and Ubuntu Server 10.04 x64</a></li>
</ol>]]></description>
			<content:encoded><![CDATA[<p>There are lots of howtos about creating a MySQL cluster, and this is another one that might be useful in some special cases. In this tutorial I will create a Two nodes Master/Master  MySQL replication on Ubuntu server 10.04. I used in my cluster two nodes  HP DL380G6 with 3 hard disks 15K in RAID5 and connected to a SAN  storage via Fiber.</p>
<p><a href="http://www.hbyconsultancy.com/wp-content/uploads/2010/07/master-master.jpg"><img title="master-master" src="http://www.hbyconsultancy.com/wp-content/uploads/2010/07/master-master.jpg" alt="" width="576" height="201" /></a></p>
<p>Note: Before beginning I want to mention that master/master  configuration is great to avoid single point of failure in your system  architecture, however it will not distribute load across the nodes. I  use Mysql specifically for users management and permissions and then  distribute load via my web application.</p>
<p>Ubuntu Server install ext4 by default, which you will need to change  first to ext3, according to <a href="https://wiki.ubuntu.com/LucidLynx/ReleaseNotes#Performance%20regressions%20with%20ext4%20under%20certain%20workloads">Lucid/Lynx  release notes</a>, if you have performance-sensitive applications &#8211; the  case with a Mysql database server.</p>
<p>We will consider two nodes node1 and node2 with respectively  10.10.0.1 and 10.10.0.2</p>
<p>First install MySQL 5.1 in the two nodes :</p>
<p><code>hatem@node1$ sudo apt-get install mysql-server mysql-client</code></p>
<p>You will have to set the MySQL root password during installation.</p>
<p>Then we have to configure the first master node1</p>
<p><code>hatem@node1$ sudo vi /etc/mysql/my.cnf</code></p>
<p>comment out the line bind-address, then set a unique value to  server-id, and the masterdbname you want to replicate.</p>
<p><code>[mysqld]<br />
# bind-address           = 127.0.0.1<br />
server-id               = 1<br />
log_bin                 = /var/log/mysql/mysql-bin.log<br />
expire_logs_days        = 10<br />
max_binlog_size         = 100M<br />
binlog_do_db            = masterdbname<br />
binlog_ignore_db        = mysql<br />
binlog_ignore_db        = test</code></p>
<p>Restart database to affect changes :</p>
<p><code>hatem@node1$ sudo /etc/init.d/mysql restart</code></p>
<p>If you have a database dump you can import it in node1, however you  can just create the database masterdbname manually :</p>
<p><code>hatem@node1$ mysql -u root -p<br />
mysql&gt; CREATE DATABASE masterdbname;</code></p>
<p>We will have to grant a <em>username</em> and <em>password</em> replication permission on the node1 :</p>
<p><code>mysql&gt; GRANT REPLICATION SLAVE ON *.* TO 'user'@'%'  IDENTIFIED BY 'password';<br />
mysql&gt; FLUSH PRIVILEGES;<br />
mysql&gt; show master status;<br />
+------------------+----------+--------------+------------------+<br />
| File             | Position | Binlog_Do_DB | Binlog_Ignore_DB |<br />
+------------------+----------+--------------+------------------+<br />
| mysql-bin.000034 |      443 | masterdbname | mysql,test       |<br />
+------------------+----------+--------------+------------------+<br />
1 row in set (0.00 sec)<br />
mysql&gt; quit</code></p>
<p>Now we can move to node2 and configure the mysql server :</p>
<p><code>hatem@node2$ sudo vi /etc/mysql/my.cnf</code></p>
<p>We will have the similar config here too, but with a different  server-id</p>
<p><code>[mysqld]<br />
# bind-address           = 127.0.0.1<br />
server-id               = 2<br />
log_bin                 = /var/log/mysql/mysql-bin.log<br />
expire_logs_days        = 10<br />
max_binlog_size         = 100M<br />
binlog_do_db            = masterdbname<br />
binlog_ignore_db        = mysql<br />
binlog_ignore_db        = test</code></p>
<p>Restart mysql</p>
<p><code>hatem@node2$ sudo /etc/init.d/mysql restart</code></p>
<p>Finally we can connect to node2 server and setup the first  replication :</p>
<p><code>hatem@node2$ mysql -u root -p<br />
mysql&gt; CREATE DATABASE masterdbname;<br />
mysql&gt; CHANGE MASTER TO MASTER_HOST='10.10.0.1', MASTER_USER='user',  MASTER_PASSWORD='password', MASTER_LOG_FILE='mysql-bin.000034',  MASTER_LOG_POS=443;</code></p>
<p>Notice that MASTER_LOG_FILE and MASTER_LOG_POS values are from  previous show master status result.</p>
<p>The first replication is done we can start the slave :</p>
<p><code>mysql&gt; START SLAVE;<br />
mysql&gt; show slave status \G<br />
*************************** 1. row ***************************<br />
Slave_IO_State: Waiting for master to send event<br />
Master_Host: 10.10.0.1<br />
Master_User: user<br />
Master_Port: 3306<br />
Connect_Retry: 60<br />
Master_Log_File: mysql-bin.000034<br />
Read_Master_Log_Pos: 261<br />
Relay_Log_File: Node2-relay-bin.000002<br />
Relay_Log_Pos: 406<br />
Relay_Master_Log_File: mysql-bin.000034<br />
Slave_IO_Running: Yes<br />
Slave_SQL_Running: Yes</code></p>
<p>If the Slave_IO_Running and Slave_SQL_Running are Yes it mean your  first replication is done and you can create a simple table :</p>
<p>At Node1 (actual master) create a table :</p>
<p><code>mysql&gt; use masterdbname;<br />
mysql&gt; create table testmaster21 (mid int(11) auto_increment, PRIMARY  KEY (mid)) Engine=MyISAM;</code></p>
<p>Check available tables at node2 (actual slave)  &#8230; Tada :</p>
<p><code>mysql&gt; show tables;<br />
+------------------------+<br />
| Tables_in_masterdbname |<br />
+------------------------+<br />
| testmaster21           |<br />
+------------------------+<br />
1 rows in set (0.00 sec)</code></p>
<p>Now the second part is very easy, we need first to check master  details in node2 :</p>
<p><code>mysql&gt; show master status;<br />
+------------------+----------+--------------+------------------+<br />
| File             | Position | Binlog_Do_DB | Binlog_Ignore_DB |<br />
+------------------+----------+--------------+------------------+<br />
| mysql-bin.000034 |      261 | masterdbname | mysql,test       |<br />
+------------------+----------+--------------+------------------+<br />
1 row in set (0.00 sec)</code></p>
<p>Then add in node1 :</p>
<p><code>mysql&gt; CHANGE MASTER TO MASTER_HOST='10.10.0.2',  MASTER_USER='user', MASTER_PASSWORD='password',  MASTER_LOG_FILE='mysql-bin.000034', MASTER_LOG_POS=261;<br />
mysql&gt; START SLAVE;<br />
mysql&gt; show slave status \G<br />
*************************** 1. row ***************************<br />
Slave_IO_State: Waiting for master to send event<br />
Master_Host: 10.10.0.2<br />
Master_User: user<br />
Master_User: user<br />
Master_Port: 3306<br />
Connect_Retry: 60<br />
Master_Log_File: mysql-bin.000034<br />
Read_Master_Log_Pos: 261<br />
Relay_Log_File: Node1-relay-bin.000002<br />
Relay_Log_Pos: 406<br />
Relay_Master_Log_File: mysql-bin.000034<br />
Slave_IO_Running: Yes<br />
Slave_SQL_Running: Yes</code></p>
<p>To test it you can drop table from Node2 and make sure the table is dropped on node1 too.</p>
<p>That&#8217;s all !</p>


<p>Related posts:<ol><li><a href='http://www.hbyconsultancy.com/blog/shared-ocfs2-partition-on-ubuntu-server-10-04-x64.html' rel='bookmark' title='Permanent Link: Shared OCFS2 partition on Ubuntu Server 10.04 x64'>Shared OCFS2 partition on Ubuntu Server 10.04 x64</a></li>
<li><a href='http://www.hbyconsultancy.com/blog/two-nodes-load-balance-and-failover-with-keepalived-and-ubuntu-server-10-04-x64.html' rel='bookmark' title='Permanent Link: Two nodes Load balance and Failover with keepalived and Ubuntu Server 10.04 x64'>Two nodes Load balance and Failover with keepalived and Ubuntu Server 10.04 x64</a></li>
</ol></p>]]></content:encoded>
			<wfw:commentRss>http://www.hbyconsultancy.com/blog/create-a-master-master-mysql-replication-ubuntu-server-10-04-x64.html/feed</wfw:commentRss>
		<slash:comments>2</slash:comments>
		</item>
		<item>
		<title>Open Source, Closed Source, We are not enemies</title>
		<link>http://www.hbyconsultancy.com/blog/open-source-closed-source-are-not-ennemie.html</link>
		<comments>http://www.hbyconsultancy.com/blog/open-source-closed-source-are-not-ennemie.html#comments</comments>
		<pubDate>Sun, 04 Jul 2010 06:47:28 +0000</pubDate>
		<dc:creator>hatem</dc:creator>
				<category><![CDATA[Blog]]></category>
		<category><![CDATA[BPM]]></category>
		<category><![CDATA[e-Government]]></category>
		<category><![CDATA[Interoperability]]></category>
		<category><![CDATA[Open Source]]></category>
		<category><![CDATA[Web Applications]]></category>

		<guid isPermaLink="false">http://www.hbyconsultancy.com/?p=312</guid>
		<description><![CDATA[Every time I read a paper about open source I can feel how it&#8217;s trying hard to oppose to closed source or proprietary software. People in the closed source side seems to take things differently : &#8221; we are doing business and we have to generate sales, closed or open doesn&#8217;t matter&#8221;. I&#8217;m not going [...]


Related posts:<ol><li><a href='http://www.hbyconsultancy.com/blog/say-yes-to-open-source.html' rel='bookmark' title='Permanent Link: Say YES to Open Source'>Say YES to Open Source</a></li>
<li><a href='http://www.hbyconsultancy.com/blog/open-source-evangelism.html' rel='bookmark' title='Permanent Link: Open Source Evangelism'>Open Source Evangelism</a></li>
<li><a href='http://www.hbyconsultancy.com/blog/e-government-as-platform.html' rel='bookmark' title='Permanent Link: e-Government as Platform'>e-Government as Platform</a></li>
</ol>]]></description>
			<content:encoded><![CDATA[<p>Every time I read a paper about open source I can feel how it&#8217;s trying hard to oppose to closed source or proprietary software. People in the closed source side seems to take things differently : &#8221; we are doing business and we have to generate sales, closed or open doesn&#8217;t matter&#8221;.</p>
<p>I&#8217;m not going to defend open source, because there are enough arguments, technologies, and solutions that are very competitive and no closed technology can compete with them, however my only take is only for education.</p>
<p>Why most education systems &#8220;impose&#8221; the use of proprietary solutions ? I did not said that MS Office is bad, but do not force me to use this product. Why teaching kids how to use Windows, Word, Excel, PowerPoint&#8230; then Access for advanced users !</p>
<p>As a student I&#8217;ll have probably to write text, reports, table sheets, presentations &#8230; I don&#8217;t think this deserve to be taught separately and marked in the final exams ! People behind education programs are sometimes old fashion and love to stick to old stuff they learned, or just prefer the solution of facility if it&#8217;s about teaching few basic features documented and sponsored by the software provider.</p>
<p>The only good things is probably teaching Java as main programming language in most universities and colleges, and my apologies for those who are stick to Pascal &#8220;forever&#8221;. It&#8217;s okay to teach .NET but don&#8217;t stick to Visual Studio and Windows environment, there is Mono for people who don&#8217;t know it which provide an open source alternative.</p>
<p>In business, the choice of open or closed technology is not a matter of evangelism. I had the chance probably to work on a large open source solution using a closed source database for a government institution, and actually another new project which is based mainly on closed technology and integrated with the open source solution.</p>
<p>The core business solution is stable for one year already, and is mainly powered by open source technologies (operating systems, web server, web application), even the closed source database is running on an open source OS.</p>
<p>Getting the new project running with closed technology doesn&#8217;t much matter for me &#8211; even if I won&#8217;t recommend it if I was asked before. Making the business evolute is my only concern, and integration was really made easy between the two technologies.</p>
<p>My only worries today is about the capabilities of the closed technology to support a high number of users, and high traffic during peak times. In the same time I&#8217;m not much worried, because the core business is in the safe side, if the new implementation for a reason or another fail, only the &#8220;closed source&#8221; part of the system will fail and not the entire system.</p>
<p>Businesses and consultancies have the choice, most of the time, but they are mostly interested to work behind a giant company name to make sure that their solution will be accepted. By experience selling a custom made solution (powered by open source technologies) is much more profitable than reselling licenses of any available software, but the lack of experience and confidence doesn&#8217;t give much choices.</p>
<p>I think I have answered a key question here : small companies cannot compete with open source software, get behind a large company name and you will be able to sell any open source solution at a very competitive price compared to closed source one. Open and closed source are not enemies, it&#8217;s up to you to use one or the other, or make them both coexist.</p>


<p>Related posts:<ol><li><a href='http://www.hbyconsultancy.com/blog/say-yes-to-open-source.html' rel='bookmark' title='Permanent Link: Say YES to Open Source'>Say YES to Open Source</a></li>
<li><a href='http://www.hbyconsultancy.com/blog/open-source-evangelism.html' rel='bookmark' title='Permanent Link: Open Source Evangelism'>Open Source Evangelism</a></li>
<li><a href='http://www.hbyconsultancy.com/blog/e-government-as-platform.html' rel='bookmark' title='Permanent Link: e-Government as Platform'>e-Government as Platform</a></li>
</ol></p>]]></content:encoded>
			<wfw:commentRss>http://www.hbyconsultancy.com/blog/open-source-closed-source-are-not-ennemie.html/feed</wfw:commentRss>
		<slash:comments>0</slash:comments>
		</item>
		<item>
		<title>Speed up your PHP website, a consultant&#8217;s guide</title>
		<link>http://www.hbyconsultancy.com/blog/speed-up-your-php-website-a-consultants-guide.html</link>
		<comments>http://www.hbyconsultancy.com/blog/speed-up-your-php-website-a-consultants-guide.html#comments</comments>
		<pubDate>Tue, 27 Apr 2010 07:10:28 +0000</pubDate>
		<dc:creator>hatem</dc:creator>
				<category><![CDATA[Blog]]></category>
		<category><![CDATA[Consultant]]></category>
		<category><![CDATA[PHP]]></category>

		<guid isPermaLink="false">http://www.hbyconsultancy.com/?p=306</guid>
		<description><![CDATA[Yesterday a friend was requesting a MYSQL/PHP Consultant to speed up his websites, and today I was reading few comments and solutions on his facebook : Use APC, mem_cache for PHP, mysql query caching, use nginx instead of Apache, Use mysqli extension instead of adodb or mysql extension &#8230; etc. Of course all of these [...]


Related posts:<ol><li><a href='http://www.hbyconsultancy.com/blog/create-a-master-master-mysql-replication-ubuntu-server-10-04-x64.html' rel='bookmark' title='Permanent Link: Create a Master Master MySQL replication &#8211; Ubuntu Server 10.04 x64'>Create a Master Master MySQL replication &#8211; Ubuntu Server 10.04 x64</a></li>
</ol>]]></description>
			<content:encoded><![CDATA[<p>Yesterday a friend was requesting a MYSQL/PHP Consultant to speed up his websites, and today I was reading few comments and solutions on his facebook : Use APC, mem_cache for PHP, mysql query caching, use nginx instead of Apache, Use mysqli extension instead of adodb or mysql extension &#8230; etc. Of course all of these could be solutions, but unless you have previously located what the problem really is.<span id="more-306"></span></p>
<p>A website could be slow for many reasons, and to be able to fix it and make it run faster you have to first find problem. What you will need to know is :</p>
<p><strong>1- Understand the business :</strong></p>
<p>Before looking into PHP or MySQL, have a first look at the website(s) and answer these two questions :</p>
<p>- What is slow<br />
- Why is it slow</p>
<p>Look at the page size, images, video, flash, &#8230; your issue might be in the client side and everything else might be good.<br />
Use a tool like Firefox web developer extension and disable everything : CSS, Javascript, Images&#8230; and load the page as html to feel the difference.<br />
The business here is NOT the business logic behind the website, it&#8217;s the interface that people use to interact with the website. If everything looks okay, you can go for further server-side investigations.</p>
<p><strong>2- Understand the production environment </strong>: Operating system, PHP and MySQL versions, PHP extensions running, PHP and MySQL configurations.</p>
<p>Do not start optimizing code or database unless you know the exact problem, otherwise you have to start by understanding and optimizing the working environment. Some tweaks in PHP and MySQL settings might fix your problem and you won&#8217;t probably need to dig deeply into any code.</p>
<p><strong>3- Understand the code : </strong></p>
<p>If the websites you are looking to optimize are using a common framework or CMS, let&#8217;s say Drupal, WordPress, Zend Framework, Symfony&#8230; such websites you can deal with separetely  since most optimization issues should be known and you will easily find your way to speed things up.</p>
<p>Otherwise, for custom code and custom application development, start with debugging and focus firstly on the database. That&#8217;s where most slow issues come from, especially if the concerned website was running fast in the beginning then become slow over the time due to database size going bigger, query &amp; index issues&#8230;</p>
<p><strong>4- Debug :</strong> detect and locate bottlenecks</p>
<p>Debuggers are your friends here, whatever the environment you are running. Answer again previous questions (What and Why) but this time with relation to the code itself.</p>
<p>You might consider running a stress test here to simulate real working environment. Such tests could help you understand more networking issues and make it easier to locate slowliness based on real scenarios. Test should also simulate real environment, and you will need to save speed result for comparison later.</p>
<p><strong>5- Solution proposal and action plan</strong></p>
<p>Now you can go ahead and set possible solutions : fixing bugs, caching code/queries, tweak your settings&#8230; Make sure your modifications will not affect other working website in the same environment &#8211; if any is somehow related or linked.</p>
<p>Finally you can run previous stress test with new configuration and updates to see the difference in speed gained. A report of previously mentioned steps should be written including result.</p>


<p>Related posts:<ol><li><a href='http://www.hbyconsultancy.com/blog/create-a-master-master-mysql-replication-ubuntu-server-10-04-x64.html' rel='bookmark' title='Permanent Link: Create a Master Master MySQL replication &#8211; Ubuntu Server 10.04 x64'>Create a Master Master MySQL replication &#8211; Ubuntu Server 10.04 x64</a></li>
</ol></p>]]></content:encoded>
			<wfw:commentRss>http://www.hbyconsultancy.com/blog/speed-up-your-php-website-a-consultants-guide.html/feed</wfw:commentRss>
		<slash:comments>2</slash:comments>
		</item>
		<item>
		<title>YEFI e-Government Interoperability Framework</title>
		<link>http://www.hbyconsultancy.com/blog/yefi-e-government-interoperability-framework.html</link>
		<comments>http://www.hbyconsultancy.com/blog/yefi-e-government-interoperability-framework.html#comments</comments>
		<pubDate>Wed, 24 Mar 2010 07:15:32 +0000</pubDate>
		<dc:creator>hatem</dc:creator>
				<category><![CDATA[Blog]]></category>
		<category><![CDATA[e-Government]]></category>
		<category><![CDATA[Interoperability]]></category>
		<category><![CDATA[Yesser]]></category>

		<guid isPermaLink="false">http://www.hbyconsultancy.com/?p=294</guid>
		<description><![CDATA[Part of Yesser e-Gov program an interoperability framework called YEFI. The framework concern any entity that exchange data and integrate services for the consumption and benefit of the public. From yesser website, YEFI is defined as &#8220;a unified framework to implement e-government. It includes cross &#8211; governmental specifications and policies, to enable cross &#8211; governmental [...]


Related posts:<ol><li><a href='http://www.hbyconsultancy.com/blog/say-yes-to-open-source.html' rel='bookmark' title='Permanent Link: Say YES to Open Source'>Say YES to Open Source</a></li>
</ol>]]></description>
			<content:encoded><![CDATA[<p><a href="http://www.hbyconsultancy.com/wp-content/uploads/2010/03/yesser-logo.gif"><img class="alignleft size-full wp-image-295" title="yesser logo" src="http://www.hbyconsultancy.com/wp-content/uploads/2010/03/yesser-logo.gif" alt="" width="226" height="108" /></a>Part of Yesser e-Gov program an interoperability framework called YEFI. The framework concern any entity that exchange data and integrate services for the consumption and benefit of the public.</p>
<p>From yesser website, YEFI is defined as &#8220;a unified framework to implement e-government. It includes cross &#8211; governmental specifications and policies, to enable cross &#8211; governmental integration and facilitate G2G transactions and data sharing.&#8221;<span id="more-294"></span></p>
<p>While reading deeply about YEFI, I noticed that the framework cover almost everything we need &#8211; for integration purpose at least &#8211; between different government institutions. The common part of the schemas between different ministries is almost the same : Address, ID number, place &amp; date of birth, Last and first name, while they keep different names.</p>
<p>The creation of data catalog is the first step to get these data defined, where every entry should include as fields : Name, Business information, Format, Validation, Verification, XML Schema IDs, Values, Default value, Owner, Version and Acceptance date. The data schema will be in this case the collection of all these elements defined in data catalog.</p>
<p>The schema should include meta data; if we are sending person&#8217;s data for example the schema should start with meta data as : Business object created, Business object updated, owner, data schema ID, Global unique object ID; then followed by person data as : Person birth place &amp; date, First &amp; Last name &#8230; etc.</p>
<p>A central repository for the standardized schemas is required to handle persistent data of oudated schemas. As every institution data might evolve over the years, and data schema will be updated; a repository should be available to document schemas versions and conversion rules.</p>
<p>The recommended integration technologies standards are : WebServices for middleware, UTF-16 for character sets, XML/XSL and XML schema, and finally RDF as resource description framework.</p>
<p>More details on YEFI program and technical documentation are available at <a href="http://www.yesser.gov.sa/english/YEFI.asp">http://www.yesser.gov.sa/english/YEFI.asp</a></p>


<p>Related posts:<ol><li><a href='http://www.hbyconsultancy.com/blog/say-yes-to-open-source.html' rel='bookmark' title='Permanent Link: Say YES to Open Source'>Say YES to Open Source</a></li>
</ol></p>]]></content:encoded>
			<wfw:commentRss>http://www.hbyconsultancy.com/blog/yefi-e-government-interoperability-framework.html/feed</wfw:commentRss>
		<slash:comments>0</slash:comments>
		</item>
		<item>
		<title>Efficient Edge 2010 &#8211; Sun Oracle Wipro Conference</title>
		<link>http://www.hbyconsultancy.com/blog/efficient-edge-2010-sun-oracle-wipro-conference.html</link>
		<comments>http://www.hbyconsultancy.com/blog/efficient-edge-2010-sun-oracle-wipro-conference.html#comments</comments>
		<pubDate>Wed, 03 Mar 2010 19:29:42 +0000</pubDate>
		<dc:creator>hatem</dc:creator>
				<category><![CDATA[Blog]]></category>
		<category><![CDATA[Datacenter]]></category>
		<category><![CDATA[IT infrastructure]]></category>
		<category><![CDATA[KAUST]]></category>
		<category><![CDATA[Oracle]]></category>
		<category><![CDATA[Storage]]></category>
		<category><![CDATA[Sun]]></category>
		<category><![CDATA[Virtualization]]></category>
		<category><![CDATA[Wipro]]></category>

		<guid isPermaLink="false">http://www.hbyconsultancy.com/?p=141</guid>
		<description><![CDATA[I assisted today at the Efficient Edge 2010 conference organized by Wipro in a partnership with Sun/Oracle. The conference was held today in Intercontinental Jeddah from 8h30 to 13h30. Sincerely beside the commercial aspect of the conference, it was really very informative, and it was a pleasure to meet few names from Sun Microsystems mainly Iyad Al-Bukhari, Andy Clark, and Thomas Bretscher.


No related posts.]]></description>
			<content:encoded><![CDATA[<p>I assisted today at the Efficient Edge 2010 conference organized by Wipro in a partnership with Sun/Oracle. The conference was held today in Intercontinental Jeddah from 8h30 to 13h30. Sincerely beside the commercial aspect of the conference, it was really very informative, and it was a pleasure to meet few names from Sun Microsystems mainly Iyad Al-Bukhari, Andy Clark, and Thomas Bretscher.<span id="more-141"></span></p>
<p>The first session was about Sun SPARC enterprise servers, and I liked the idea of supporting mix of processors versions in the M4000, M500 and M9000 (the M series servers) &#8211; and it&#8217;s only Sun that offer this compared to IBM and HP. Some benchmarks also in this session and was very informative; also covered different aspect of the M-Series and CMT servers for single-threaded and multi-threaded.</p>
<p>In the second session the Datacenter Architect talked about Datacenter efficiency through virtualization. Andy talked about different level of virtualization Servers, Storage, and Network, in addition to virtualization tool Sun VDI software and Sun Ops Center. The business case in the end was very interesting showing 14% ROI in the first year only. There is something I noticed that virtualization tools are available free with Open Solaris, and coming soon to Solaris; and you don&#8217;t have to pay more to use them ! It&#8217;s something like when you buy a new car, then you try to use the Radio and they tell you have to pay more for this feature. Well, I totally agree with Andy and find it really ridiculous to do this; especially that I have a similar problem with HP Storage and the way they limit licencing to a fixed number of TB &#8211; so to add more capacity we&#8217;ll need a new license.</p>
<p>In the last session, Thomas talked about Sun Storage 7000 Unified Storage Systems. Almost all speakers was insisting on the ZFS &#8211; the famous Sun File Systems; obviously it was mentioned more frequently with the storage session. I liked especially the Enterprise SSDs, which eliminates storage bottlenecks. The ZFS pool guarantee a better performance for writes via the ZIL pool and reads via L2ARC pool &#8211; both connected to SSDs in addition to a third pool with cheap HDs.</p>
<p>The last session was about Sun FlashFire; also by Thomas. There was a case study of the KAUST presented by Mohammed Abdel Aal; Manager KAUST IT Computing Infrastructure. Mr Mohammed talked more about KAUST research strategy, but there is something that I liked most in KAUST IT requirements &#8220;No legacy systems, no legacy thinking&#8221;.</p>
<p>Overall it was great meeting IT professionals. I have already worked with Sun servers and Solaris, but I don&#8217;t have previously good knowledge of their hardware architecture. I was only familiar with ZFS and Open Solaris.</p>
<p>I didn&#8217;t stay for the lunch, but I expect it to be great too.</p>


<p>No related posts.</p>]]></content:encoded>
			<wfw:commentRss>http://www.hbyconsultancy.com/blog/efficient-edge-2010-sun-oracle-wipro-conference.html/feed</wfw:commentRss>
		<slash:comments>0</slash:comments>
		</item>
		<item>
		<title>e-Government as Platform</title>
		<link>http://www.hbyconsultancy.com/blog/e-government-as-platform.html</link>
		<comments>http://www.hbyconsultancy.com/blog/e-government-as-platform.html#comments</comments>
		<pubDate>Sat, 30 Jan 2010 09:09:44 +0000</pubDate>
		<dc:creator>hatem</dc:creator>
				<category><![CDATA[Blog]]></category>
		<category><![CDATA[BPM]]></category>
		<category><![CDATA[e-Government]]></category>
		<category><![CDATA[Web Platform]]></category>

		<guid isPermaLink="false">http://www.hbyconsultancy.com/?p=137</guid>
		<description><![CDATA[Ten years ago, e-Government was a term that every Government started working hardly to study, implement, and understand better how new information technologies could help making better Government services for people. Things started with legislations and laws in different countries, and step by step we started seeing countries embracing the e-Government concept 100%. Building e-Government [...]


Related posts:<ol><li><a href='http://www.hbyconsultancy.com/blog/open-source-closed-source-are-not-ennemie.html' rel='bookmark' title='Permanent Link: Open Source, Closed Source, We are not enemies'>Open Source, Closed Source, We are not enemies</a></li>
<li><a href='http://www.hbyconsultancy.com/blog/yefi-e-government-interoperability-framework.html' rel='bookmark' title='Permanent Link: YEFI e-Government Interoperability Framework'>YEFI e-Government Interoperability Framework</a></li>
<li><a href='http://www.hbyconsultancy.com/blog/open-source-evangelism.html' rel='bookmark' title='Permanent Link: Open Source Evangelism'>Open Source Evangelism</a></li>
</ol>]]></description>
			<content:encoded><![CDATA[<p>Ten years ago, e-Government was a term that every Government started working hardly to study, implement, and understand better how new information technologies could help making better Government services for people. Things started with legislations and laws in different countries, and step by step we started seeing countries embracing the e-Government concept 100%.<span id="more-137"></span></p>
<p>Building e-Government services is not that hard, not that easy also. There are many steps that should be followed to be able &#8211; at a certain time &#8211; to move totally from an old paper-based system, to a fully electronic system.</p>
<p>The most interesting in e-Government that you are building services for people, and people are the customers, the users, while they usually never interact during the building process &#8211; but we have many considerations to keep in mind while building for people.</p>
<p>Some interesting steps in e-Gov creation where we should focus mainly include :</p>
<ul class="secondList">
<li>Business Process Re-engineering</li>
<li>e-Government system architecture</li>
<li>Systems interoperability and scalability</li>
<li>Security : systems, network, and end-users</li>
<li>Cost consideration : hardware, software, network, development in-house, outsourcing, and management</li>
<li>Web-enabling requirements : accessibility, usability, technologies standards</li>
<li>Supporting Multiple Communication channels</li>
<li>Internal software and hardware for administration (legacy systems) : RH, Finance, DMS, Archiving, CRM, Workflow, Communication (Email, Voice, IM)</li>
</ul>
<p>It is interesting to turn Governments process into e-Services, but building platforms that support different business process levels G2G, G2B and G2C is much more interesting.</p>
<p>Some experts started talking about Government 2.0, or Government as Platform. A great concept especially since it allows anybody to build government services or integrate it with other already established business process elsewhere.</p>
<p>The problem is that building platforms is totally different from building services, and IT developers/architects used to create web applications that do not scale at the Government level. One of these problems is the execution of the <a href="http://www.julianbrowne.com/article/viewer/brewers-cap-theorem">CAP theorem</a> in Governments web applications. The fact that we require &#8220;strong consistency&#8221; in e-Government is a major limitation for high availability in web platforms.</p>
<p>&#8220;eventual consistency is an  eventual solution, but implement it correctly is a major obstacle. The reason why developers/architects have to better understand the best practices for building eventually consistent systems, in highly available environments.</p>


<p>Related posts:<ol><li><a href='http://www.hbyconsultancy.com/blog/open-source-closed-source-are-not-ennemie.html' rel='bookmark' title='Permanent Link: Open Source, Closed Source, We are not enemies'>Open Source, Closed Source, We are not enemies</a></li>
<li><a href='http://www.hbyconsultancy.com/blog/yefi-e-government-interoperability-framework.html' rel='bookmark' title='Permanent Link: YEFI e-Government Interoperability Framework'>YEFI e-Government Interoperability Framework</a></li>
<li><a href='http://www.hbyconsultancy.com/blog/open-source-evangelism.html' rel='bookmark' title='Permanent Link: Open Source Evangelism'>Open Source Evangelism</a></li>
</ol></p>]]></content:encoded>
			<wfw:commentRss>http://www.hbyconsultancy.com/blog/e-government-as-platform.html/feed</wfw:commentRss>
		<slash:comments>0</slash:comments>
		</item>
	</channel>
</rss>

<!-- Performance optimized by W3 Total Cache. Learn more: http://www.w3-edge.com/wordpress-plugins/


Served from: www.hbyconsultancy.com @ 2010-09-08 11:49:48 -->