Sequoia JDBC Database Driver :: GOTCHAs

So this post is another along the more technical support side. I wanted to create a log of GOTCHAs i've encountered using Sequoia's High Availability database library for my personal reference and to help others with the same problem. sequoia.continuent.org

GOTCHA's
Symptom: My Java Client connects to the virtual database, executes a query, but then sequoia throws a ProtocolException without any information about what went wrong.

Debug: To debug append the 'debugLevel=debug' to your connection URL. You should see information about the connection being made and the authentication working. You should also have the console output of your controllers visable while debugging.

Solution: If you notice WARN messages saying your connection has been closed, it is often due to an incompatible Sequoia driver jar. Assuming you put this driver into your tomcat/lib folder (or where ever) just replace this with the sequoia-driver.jar that matches the installation of sequoia you are using. These drivers appear to not be compatible with any other version than the one they are packaged with!

Web 3.0 Opportunities

It is no secret that the Web is evolving at a rapid pace. It is also common knowledge that the vision of the 'Web 3.0' is based on the representation of data, and often referred to as The Semantic Web. Now if you are currently a web 2.0 developer without much or any knowledge about the semantic web and the technologies recommended to power it, I suggest reading up on a few of the W3C recommendations for RDF and OWL.

Basically, these and other specifications, based entirely on XML, provide information about information (data about data, i.e. metadata). Web Documents can/should be represented in XML styled with XSL and meaning represented by RDFs. This provides the ability for systems to become intelligent towards content, opening up immense opportunities.

Organizations no longer need custom applications to publish their data. Why not simply publish XML documents with detailed RDFs and let already developed applications based on these specifications provide presentation? Now of course this is not a new idea. Company A has products, company B provides services to display these products on the Web. The only difference is now it standardizes and makes the design process for all web publications homogeneous. This loosely coupled Data and Presentation idea makes powerful tools and ideas like mashups insanely easy.

The ability to consolidate tools and presentation libraries must appeal to anyone with half a brain (as long as it isn't devoted towards securing niche markets and complicating the world to make full service sales that make event the Gates wince).

Adding meaning to Web Data and keeping the presentation layer separate open up so many possibilities and will inspire even more explosion of the services and usage of the Internet. More intelligent systems will become available and computing will continue to evolve to become more useful and automated.

I would love to hear opinions on the Semantic Web ideals and also about possibilities it creates. (Please criticize as well!)

Load Balancing With Apache 2.2.8 and Tomcat 6

This blog entry is in an effort to help me and others remember how to setup a small cluster using Apache httpd 2.2.8 and Tomcat 6. It is meant to be used in conjunction with the Apache documentation and possibly other resources. If you are familiar with the technologies then this short post might work as a standalone configuration example.

Setup
Apache HTTPD Server 2.2.8
3 Tomcat 6.0.16
Windows XP
No FIREWALL!!!

Load Balancer Choice
The mod_proxy_ajp was chosen as the load balancer because it provides good performance and easy setup. AJP is a simplified HTTP protocol that Tomcat and Httpd can use to communicate. You can also use the HTTP proxy module if you'd like.

Required httpd.conf Changes
[... ]

LoadModule proxy_module modules/mod_proxy.so
LoadModule proxy_ajp_module modules/mod_proxy_ajp.so
LoadModule proxy_balancer_module modules/mod_proxy_balancer.so
LoadModule proxy_connect_module modules/mod_proxy_connect.so
LoadModule rewrite_module modules/mod_rewrite.so
LoadModule status_module modules/mod_status.so

[...]

<location>
   SetHandler balancer-manager
   Order Deny,Allow
   Deny from all
   Allow from localhost
</location>

<proxy>
   Order Deny,Allow
   Allow from all
</proxy>

<ifmodule>
   <virtualhost>
       # Don't allow server to allow spam proxy usage
       ProxyRequests Off
       # Give all possible load balancing information
       ProxyStatus Full

       # Allow access to the balancer-manager using httpd instead of tomcat
       ProxyPass /balancer-manager !
       ProxyPass /logs !
     
       # Setup pass to balancer for tomcat calls
       # ProxyPass / balancer://mycluster stickysession=JSESSIONID nofailover=Off lbmethod=bytraffic timeout=1
       ProxyPassMatch ^(/.*)$ balancer://mycluster stickysession=JSESSIONID nofailover=Off lbmethod=bytraffic timeout=1
     
       # Fix redirects
       ProxyPassReverse / ajp://10.1.0.113:7009/
       ProxyPassReverse / ajp://10.1.0.113:8009/
       ProxyPassReverse / ajp://10.1.0.113:9009/
 
       # Make sure cookies are still going to work
       #ProxyPassReverseCookieDomain localhost localhost:6080
       #ProxyPassReverseCookieDomain localhost:8009 localhost:6080
       #ProxyPassReverseCookieDomain localhost:9009 localhost:6080
       #ProxyPassReverseCookiePath / /
 
       # Define cluster
       <proxy>
           BalancerMember ajp://10.1.0.113:7009 loadfactor=1 route=jvm1
           BalancerMember ajp://10.1.0.113:8009 loadfactor=1 route=jvm2
           BalancerMember ajp://10.1.0.113:9009 loadfactor=1 route=jvm3
       </proxy>
   </virtualhost>
</ifmodule>

NOTES:
- The 'route' parameter is necessary unlike I originally thought. It is appended onto the session idea in tomcat and the name must correspond to the tomcat instance being referenced by the balancer.

Tomcat Server.xml file
This cluster configuration requires minor tweaks in each tomcat instance and fields like 'jvmRoute' must match the 'route' parameter in the httpd.conf.

[...]

<Engine name="Catalina2" defaultHost="localhost" jvmRoute="jvm2">
 <Cluster className="org.apache.catalina.ha.tcp.SimpleTcpCluster"
                 channelSendOptions="7" clusterName="mgstTomcatCluster">

          <Manager className="org.apache.catalina.ha.session.DeltaManager"
                   expireSessionsOnShutdown="false"
                   notifyListenersOnReplication="true"/>

          <Channel className="org.apache.catalina.tribes.group.GroupChannel">
            <Membership className="org.apache.catalina.tribes.membership.McastService"
                        address="228.0.0.4"
                        port="45564"
                        frequency="500"
                        dropTime="3000"/>
            <Receiver className="org.apache.catalina.tribes.transport.nio.NioReceiver"
                      address="auto"
                      port="4002"
                      autoBind="100"
                      selectorTimeout="5000"
                      maxThreads="6"/>

            <Sender className="org.apache.catalina.tribes.transport.ReplicationTransmitter"
  waitforAck="true" replicationMode="pooled">
              <Transport className="org.apache.catalina.tribes.transport.nio.PooledParallelSender"/>
            </Sender>
            <Interceptor className="org.apache.catalina.tribes.group.interceptors.TcpFailureDetector"/>
            <Interceptor className="org.apache.catalina.tribes.group.interceptors.MessageDispatch15Interceptor"/>
          </Channel>

          <Valve className="org.apache.catalina.ha.tcp.ReplicationValve"
                 filter="" statistics="true"/>

           <ClusterListener className="org.apache.catalina.ha.session.ClusterSessionListener"/>
        </Cluster>  

[...]

Questions/Comments
Feel free to post questions in the comments section, or email me if you want some help setting up a cluster like this.

Clean Web Services don't need SOAP. Let them REST!

Many technologies have a way of over-complicating simple concepts and end up tagging these unneeded complications with misused adjectives. Sometimes ignorant documents will invent their own jargon or try to promote it by calling it enterprisy. What is Enterprisy? Well, what does it make you think of? Perhaps, you think of a huge corporate board room full of silver spoon executives decided how they can get a bigger yaht, oh yeah and that the company needs to invest in some technology infrastructer. Better get that brand that is made for enterprises :/...

Now obviously I am being a little dramatic here and completely one sided. The beauty of blogs is that I can be.

Anyway, on to what I really wanted to talk about, RESTful Web Services. Now I am not a RESTful and RESTful only advocate by any means, but really RESTful is just such an elegant solution that is passed over quickly by anyone who has never laughed when they read about something being 'Enterprisy'.

First of all if you are unfamiliar with the meaning of RESTful Web Services, please google it and read about what it is and learn about its comparison to SOAP. There is enough documentation out there to help you solidify these two technologies without wasting my time trying :).

What RESTful offers is a convention based way of Web Service communication that utilizes existing HTTP protocols instead of putting verbose meta data ontop of it like in SOAP. What does this translate into? Speed, simplicity, reuse of wide spread technology... take your pick. Now, soap is definately a powerful tool these days as many coporations have invested in these tools and promotion of this method of Web Service generation making it a valuable weapon in you development arsenal. My problem is that every layer you build on you are supposed to be trading convience for performance, but in SOAP's case I just don't feel that convience factor out waying the cost. You can have the same convience without the over head in RESTful design.

Now the bad part, which is going to be the dominate technology in 5, 10, and mabye 20 years? I don't know, but I'd be willing to bet SOAP is still going to be more popular that RESTful design. Why? Because of the heavy investment made by corporations around the world. Moreover, does it really matter if you use RESTful or SOAP or something as old as COBRA? As far as a user is concerend, no fucking way. What does matter is that your solution is relatively secure and relatively inexpensive to develop and maintain. If you are the owner of your own company you have the benifit of choosing your own technologies. Unfortunately, that is a small percenage. Anyway, seeing nobody reads my posts anyway i'll stop ranting and if your that lone wolf still reading this make sure you check out RESTful Web Services.

PHP Development, the perfect framework

So I've spend many hours on many projects in PHP and I continually tweek my design concepts aiming for a generic outline that should work for any php application. This has proven to be quite a difficult task, because with every new project a new design comes to mind. Modular systems have been the focus of much of my work and my framework concepts are built accordingly. With the endlessly popular AJAX buzz i now find myself contemplating how applications will be accessed. A method that allows both traditional access and javascript access is required. It is frustrating and wasteful to continually redesign the structure of your projects.

Bottom up Design

The best way to build projects is a mixture of Bottom up and Top down design. With that in mind perhaps the best way to provide a general solution that is simple and elegant would be to provide two stages: One, a basic framework for bottom up modules that should be programmed generically enough to work anywhere; Two, a suggested method for integrating the components to provide the required features.

So now with that inmind, perhaps a module bridge system should be used to allow external applications to be loaded directly into pages, like portlets.

The Big Questions

How could one create a script to automate the process of writing bridge files? Would converting all links and forms into ajax enabled components provide a reasonable solution? Could one write an intelligent application that could convert any standard web based interface into a compatible AJAX interface that could instantly be plugged into a simple portal on another page?

So Tired...

So i've been awake for about 48 hours now, maybe some sleep would help me clear my thoughts a bit more. My major delema is the question of "IS IT REALLY WORTH THE TROUBLE TO WRITE SUCH SOFTWARE?" Surely this would not be a simple task otherwise it would be done and everyone's website would be an intelligent flexible system which could support any software that works on the web without manually modifying a bit of code...


Maybe some day my dream will come true and this will become a reality.

The World Semantics and Folksonomy

So I just read an Interesting article on slashdot, "Apple and Google to Blog the World" and found myself thinking about all the information Google is gathering though their free services, about the world. I am intrigued by the possibilities of having intimate information from billions of people all over the world about their favorite locations in the World. Tight integration of GPS devices, popular OSs, and web based repositories could allow travel agencies to promote locations based on information provided by the average joes who visit them. This is only the tip of the ice berg, by tagging and defining locations all over the world we open up a new resource for many AI applications, not to mention the immediate end-user delight of being able to read and post information about locations they keep close to their hearts.

Google's Free Services

I find myself coveting Google as they build their empire based on free services to the average user, a dream of mine before I knew it was already being done. I use Google as a Word Processor, Spreadsheet Accountant, Photo Gallery, News Portal, Search Engine, Email Client, and Blogger! I find myself dumbfounded, being able to use all of these wonderful features from anywhere for FREE! The simplistic, yet powerful layout of these applications make them more desirable to use than standard Windows/Linux, not to mention the price difference ;). Being an Open Source developer at heart I find myself wanting to use free/open software whenever possible. I admit the Microsoft office suite does have more functionality than the new Google Docs and Spreadsheets, but most of this functionality goes unused, especially by myself. Moreover; with Google I can access my documents from anywhere with an internet connection. Granted I have a laptop and I could save all my documents there and have them everywhere anyway, but with my frequent need to wipe my system out and reinstall my OS es I have begun to rely on remote storage of important files.

I also hate installing applications on my computer because it clutters my hard drive and slowly cripples the performance. I like to keep my computer lean and mean using as many remote services as possible.

Someday I think it would be great to work on a project with Google and find out how the pros collaborate on large projects in a corporate environment. My first step is to hopefully participate in the Google's summer of code.

My biggest concern is that the rumors of commercializing the Google "Web OS" and charging a fee for the services will be fulfilled. I believe if this happens they maybe able to keep some of the users hooked and make some short term profit but their long term outlook will be grim at best. I know my first instinct would be to migrate all of my work to another service provider, for example Yahoo, which is Google's biggest competitor. Many of my friends believe Google will not be able to maintain its community friendliness and inevitably will become more of a corporate monster like Microsoft. Not that I wouldn't love to have some stock from the 80's in Microsoft, but there is quite a stigma around them because of their bullish business tactics that cause many to refuse PAYING for their software. I believe there can still be morals in business or at least companies can pretend to have morals, after all Google is profiting ;P. I guess there are a few ways to look at a business' morals, after all, it is all about money in the end anyway.

My new dream is to create a wonderful service and sell it for Millions, LOL! I must study the successes of Writley, Picasa, and Blogger. Having fun writing software that helps everyone and making money, ahh, my American Dream.