When SugarCRM Case Closed

Using the cases.date_modified is not the same as the date a case is closed, as the case may have been modified after it was closed.
Thus, the query below will return the date the case was transitioned to ‘Closed’ provided the cases.status field is being audited.

select
  CONVERT_TZ(cases_audit.date_created,'+00:00','+10:00') as 'date_closed',
  accounts.name as 'account',
  cases.case_number,
  cases.name as 'case_id'
from
  cases_audit
  join cases on cases.id = cases_audit.parent_id
  join accounts on accounts.id = cases.account_id
where
  cases_audit.field_name = "status"
  and cases_audit.after_value_string = "Closed"
order by
  accounts.name asc,
  cases_audit.date_created desc;

Note, you might get more then one row returned per case if the case was re-opened, also my code above assumes a timezone of UTC+10

I happen to be in the UTC+10 timezone :).

Tracking down long running queries in SQL

I’ve had a client whose Linux Server (CentOS) gradually gets slower and slower, then falls over.

By logging into the MySQL command line, we were able to see that there were some long running queries that never end.

show processlist;

returns


| Id | User | Host | db | Command | Time | State | Info |
+———+———-+——————-+————–+———+——+————–+——————————————————————————————————+
| 2322386 | ssgadmin | 10.21.1.149:58526 | sugarcrm_ssg | Query | 2202 | Sending data | SELECT IFNULL(hr_humanresources.last_name,’’) hr_humanresources_last_name ,l3_cstm.verification_c l3 |

The key part here is that hr_humanresources_last_name has an underscore between the table name when it should be a period i.e. hr_humanresources.last_name. Knowing the table name was enough to tell me it was caused by something to do with SugarCRM that’s running on that server. But we don’t yet know what action caused these queries.

Knowing it was 2202 seconds ago from the time we ran the query we are able to pinpoint a time the action occurred. Looking through the SugarCRM log files did turn up that the error was often caused by a single user, but didn’t show up anything to help us figure out exactly what was the causal issue. Talking with that user, getting them to do what they’d done at that time didn’t turn up anything, the error wasn’t reproducing on demand.

So I turned to looking in the tracker table in the SugarCRM MySQL database for entries around the time of the error. Turns out that there is a Dashlet being loaded, that Dashlet uses a Report.

Each time I load the Dashlet or run the Report I get a corresponding long running query turn up. We’ve found our culprit, and we’re now able to recreate the report.

How long running are those queries?

At the MySQL commandline, you can enter
show processlist;
and see what processes are running.
If you’re tracking down some problematic code and you’ve got a query (or set of queries) that don’t end, consume CPU cycles and generally bog everything down to a crawl, it can be helpful to know what time you’ve run the command above. Thusly, I now run this:

show processlist; select now as current_date_time;

and it rather nicely tells me the current date/time on the MySQL server, using the Time column from the processlist, you can work out a tad more accurately when a query started, giving you a starting point to look in other logs.
Now those long running sql queries are easier to track down.

How to backup SugarCRM from the command line

From the linux command line, these commands let you backup and restore a SugarCRM database.

Update 2016-04-26: Just saw this SugarCRM KB Providing a Backup Without Sensitive Data which is really useful. Thanks Jared.

Firstly, for a proper back up of SugarCRM you’ll need two files, one containing the application files, one containing the SQL database.

First up, the backup…

Backup files:

Change to the folder you want to backup, then…

tar -zcvf CRM-BACKUP-FILES.tar.gz .

Edit: if you get an error ‘Permission Denied’ you may be trying to write to a folder you don’t have permission for, instead try writing to ~/CRM-BACKUP-FILES.tar.gz and it will likely work.
Edit: see also this StackOverflow article.

Backup sql (empty copy of the database):

mysqldump -u USERNAME -p -–no-data DATABASENAME > CRM-BACKUP-SQL.sql

Backup sql (with the data) :

mysqldump -u USERNAME -p DATABASENAME > CRM-BACKUP-SQL.sql

Then you can ‘tar’ the .sql file with

tar -zcvf DATABASENAME-mysql.tar.gz DATABASENAME-mysql.sql

Backup just a single table

mysqldump db_name table_name | gzip > table_name.sql.gz

If you only want the database schema, then in the SugarCRM web application you can do the following:

  • Admin
  • Diagnostic Tool
  • db schema
  • Download that file

Then for the restoration…

Restore files (to current folder):

tar -zxvf CRM-BACKUP-FILES.tar.gz

Restore sql:

mysql -u USERNAME -p DATABASENAME < CRM-BACKUP-SQL.sql

Restore just a single table

gunzip < table_name.sql.gz | mysql -u username -p db_name

These SugarCRM Knowledge Base articles may also be of use:

Who should pay for Microsoft SQL installations that go wrong?

ah, SQL humorThe day is almost over, so I’ll fill you in on some of what occurred today, as it relates directly to this post by Mike at GLComputing. This was originally written as a comment to his post and kinda grew, so I’m posting it here in full.

SQL 2008 Express R2 has finally let me install a default SQLEXPRESS instance but I still can’t get a custom instance of ACT7 working. Who should be paying for the time it’s taking me to work out this issue?

I don’t yet have a complete answer to that question.

Imagine for a moment that I buy a brand new car, from a dealership. The car has trouble starting. So I go back to the dealership and ask them to fix it. They do so. I as a client go home happy.

But what’s really occurred? Well, the dealership gets a mechanic to look at it; he determines that a component if faulty, he replaces it with one off the spare part shelf in the dealership.

The dealership doesn’t want to wear the cost of the mechanics time or the cost of the part, so they put in a warranty claim to the manufacturer for the time taken by the mechanic and the cost of the part.

The manufacturer pays up, occasionally they audit the dealership to make sure fraud isn’t occurring. The manufacturer actually obtained the faulty component from a supplier. They then make a claim to the supplier for the costs.

The supplier pays up.

I know this because many years ago (early ’90’s) I was a Warranty Manager for a car dealership.

Should the same process apply in the software industry?

Is it the fault of the reseller/dealership that software/component failed?

Is it the fault of the software house/manufacturer that the software/component failed?

Is it the fault of the supplier that the software/component failed?

Of course the initial supplier will argue that they make their product to stringent standards, and they can’t account for all the possible variables of other hardware and environmental conditions.

And of course the software house/manufacturer will say they took all appropriate steps.

And also, of course, the reseller/dealership will say they aren’t to blame either.

And the customer, well, the customer is never wrong, right?

Getting a little more specific, in my case today, the client has all good name brand equipment, setup by a reputable IT firm with a solid reputation. I couldn’t fault either their spec’s or configuration. I know the amount of effort I’ve put into this today. I know how much effort I’ve put into making SQL installs go smoothly, to the point where they mostly do go smooth for me, but sometimes, like today, they go very very wrong.

And so yes, I lay the fault with the remaining two players. Me, the guy on the pointy end of the issue, who gets to look like an idiot in front of a customer because he can’t make shrink wrapped software work on name brand computers. [Do you think I’m a little cranky? hint, I am.]

You see, the vendor here (Sage) have a product they sell (Sage ACT!) that uses a database in the back end (Microsoft SQL Server 2008 Express R2). The vendor (Sage) has chosen to use that product, use that version of the product. That choice means an implicit responsibility to issues using their product with the 3rd party product. Let me say it clearly, “Sage, you chose to use Microsoft SQL 2008 Express R2, which makes you half responsible.”

The 3rd party, the supplier of that component, Microsoft, have chosen to update their product, and sometimes that transition hasn’t gone smoothly for them, but nonetheless they have updated on a semi-regular basis (although not yearly, and the topic of frequency will be the subject of another discussion). They provide help via their KB articles and revert to the line “too many other environmental factors, not our problem”.

Reminds me of a joke I heard years ago:

A helicopter was flying around above Seattle when an electrical malfunction disabled all of the aircraft’s electronic navigation and communications equipment. Due to the clouds and haze, the pilot could not determine the helicopter’s position and course to fly to the airport. The pilot saw a tall building, flew toward it, circled, drew a handwritten sign, and held it in the helicopter’s window. The pilot’s sign said “WHERE AM I?” in large letters. People in the tall building quickly responded to the aircraft, drew a large sign and held it in a building window. Their sign read: “YOU ARE IN A HELICOPTER.” The pilot smiled, waved, looked at her map, determined the course to steer to SEATAC airport, and landed safely. After they were on the ground, the co-pilot asked the pilot how the “YOU ARE IN A HELICOPTER” sign helped determine their position. The pilot responded “I knew that had to be the Microsoft building because, like their technical support, online help and product documentation, the response they gave me was technically correct, but completely useless.” – Thanks to Alun for the source link.

In my view, if a car had, say a throttle problem, the manufacture would be sorting out the problem quick smart. Sage, you need to compensate the people in the front line, in the trenches. Go hit Microsoft up if you’re not happy about it. That’s what the car manufacturers do. It works for them so don’t tell me it can’t work.

But, Sage, I’m not hearing much from you. And that’s concerning to me because in today’s world, the world of 2011, the internet, social media and a with your own social media presence, to not hear much at all is to hear all the other dissenting voices, to hear the competition.

Now honestly, the competition to Sage ACT! is woeful. Seriously, it is. ACT! is a great product, it’s flexible, customisable, and at least 11 other herbs and spices, all of which are pure goodness (honestly, it’s a lot more than 11). So here’s a hint to the competition, if you want a good CRM product, imitate ACT!.

Want to know one of its weaknesses? It’s reliance on product that doesn’t install properly. Achilles only had one heel that gave him trouble. Most, but not all, of the competition have figured this out, and allow either multiple backend databases to be used (MSSQL/MySQL/Oracle/PostgreSQL and others).

Now if you agree with me, let me know, if you don’t let me know as well, because as those that know me can attest, if you can prove your point, I’ll change. Of course, if you don’t give two hoots, then I guess I won’t be hearing from you. Either way, I’m gonna go hold my teddy bear and sing myself to sleep, hoping that the SQL install nightmare doesn’t plague me tonight. I’m gonna need some sleep to go fight this dragon again.

SQL 2005 Versions

I have to thank Kevin Chieff (ACT! guru) for pointing me to this link on how to tell the difference between versions of Microsoft SQL 2005 – and the MS page it points to re the SERVERPROPERTY (Transact-SQL) command.

The number of times I’ve had to work out just which version of SQL is running is amazing. The difficulty in doing so is amazing. There ought to be a simple utility that does it all.

So the two links above will get used a lot in conjuction with my previous post re determining SQL versions.

Evolving the ACT

It is now two weeks in to my new digs at Evolution Marketing – and it sure is fun.

Primarily I’ve been learning how to drive ACT!, software to take care of your contact management. Yesterday and today I’ve been playing with custom tables in the ACT! MS-SQL database. While ACT! looks after contacts and tasks, meetings & calls to do with your contacts really well, some clients would like to be able to do even more, like handle job sheets etc. Which is the reason for the custom tables. These allow us to track all sorts of other information and connect them to contacts & companies.

I’ve also setup a Blackberry Bold simulator and a Windows Mobile 6 Emulator, this allows me to experience the same things as our clients, notably HandHeldContact, which allows you to sync multiple calenders, notes, histories etc to your phone – loads more than MS Exchange alone allows.

To paraphrase the paragraphs above, I’m having fun learning new stuff!

The other staff at Evolution are great to work with, the coffee is good and all the tech toys I’ve asked for have been delivered, with the exception of the telephone headset, but it is on the way so they’re forgiven ;-). Woot!

Update 20th Oct: Got the telephone headset on Friday afternoon. Cool.