Jess Portnoy is a prolific PHP developer and open source geek with lots of helpful data and web data utilities on SourceForge and GitHub.
I was vaguely familiar with Jess's work from various tech talks that she's given, which usually attracted my attention because of her affiliation with the web multimedia platform Kaltura.
Her upcoming talk at LinuxFest Northwest is all about PHP monitoring, and given the kind of traffic Kaltura deals with, there are likely few people as familiar with the subject as Jess.
What is JaM?
JaM is a PHP monitoring system that supports storing PHP events (fatal errors, warnings, notices, basically everything listed here) into different storage backends. The events can later be retrieved from backends that implement PHP_JAM_GET_FUNC().
Is JaM something that is primarily useful for big heavyweight sites, or could someone using just a little bit of PHP to augment a site benefit from it as well?
I believe every PHP-based system can benefit from using JaM.
Naturally, the bigger you are, the more likely you are to have issues you are unaware of and the more important monitoring becomes. However, seeing how JaM requires very little initial setup and is only triggered when erroneous events occur, I feel smaller projects can benefit from it too.
JaM started life, I gather, as the project Aware. How did you get involved with it, and what does JaM deliver that was missing from Aware?
That's a rather good example of how amazing FOSS is, actually.
I started thinking about JaM following a conversation I had with a colleague of mine. His team had a fatal error in their code because the person who wrote it tested it on PHP 5.4, whilst the production site runs on PHP 5.3. He told me his team looked at the logs and couldn't find any entries indicating an issue.
The reason for that was that someone set the error_log PHP directive to a different log location, so the error did not go to Apache's error log (which is where the team was expecting to find it).
So, I started thinking about creating a solution that does not require log parsing at all, but instead plugs directly into the Zend Engine and overrides zend_error_cb() to report the errors by email and other methods and only then write to the error log, which is what the original zend_error_cb() function does.
I already knew of one such solution by Zend, called Zend Monitor. But Zend Monitor is not free software, and has dependencies on other Zend proprietary components. I wanted a FOSS solution.
In my opinion, the first thing you need to do when you get an idea for a new project is check whether or not it already exists, and that's exactly what I set out to do. It wasn't very easy to find, but using a combination of "zend_error_cb()" and other keywords, I got to PHP Aware by Mikko Koppanen.
Looking at the project's README, it felt like it would be a great place to start from, as it already did most of what I wanted to achieve and was written in a very modular way, which is always good.
When I tried to build it, however, it did not compile. That was because it was last worked on in 2011 and the PHP engine has changed quite a bit since.
And so I ported it to support newer PHP versions, added the ElasticSearch backend as per my original plan, and submitted a pull request.
I reached out to Mikko asking if he'd like to merge it. Mikko said that while he is happy someone has taken interest in the project, he was no longer working on it or PHP in general, for that matter, and that he was more than happy for me to take ownership.
And this is how JaM was born.
Kaltura, presumably, uses JaM. Is it purely a backend technology for sys admins who love to look at logs, or is there a pretty frontend for site admins who love to look at charts and graphs?
Kaltura indeed uses JaM to monitor on our dev and testing ENVs; JaM was my project for the company hackathon we held this year. It's currently used by some of our teams, including my own, and I hope it will be adopted by others as well.
The main target audience for JaM is developers because JaM events include the full stack trace for the event, thus allowing them to understand the flow that triggered it.
Sys admins can also gain insights by reviewing the events as it can often indicate issues with supporting services and not necessarily in the code itself. For example, let's say your application uses MySQL and MySQL is currently down; this is likely to trigger multiple events since your application cannot work without it, and it's up to the sys admin to correct that.
As far as front ends and graphs go, since JaM knows how to use ElasticSearch for storing events, you can utilize Kibana to get visual representations of the events, run queries, and create graphs quite easily.
In addition, because inserting data into ElasticSearch can be done by making simple HTTP requests, passing along JSON formatted data, the ElasticSearch backend for JaM can be used to insert events to other systems that can accept data this way as well.
ElasticSearch was one of your original ideas for the project. What does it do for JaM?
The main goal is to make searching faster. However, like I mentioned in relation to your question about graphs and other data visualizations, by using Kibana, you can get quite a lot of added value from your data without having to write so much as one line of code.
What does JaM offer that other monitoring systems do not?
Because JaM gets the event data directly from the Zend Engine, the necessity of parsing logs or creating APIs to report your application's errors to a monitoring system is completely avoided.
Parsing big files is slow and sometimes difficult because you need your algorithm to determine whether or not the line it is now looking at is actually an error or not and can often get false alarms or else, miss an error due to faulty parsing patterns.
Looking at parsed results can also be cumbersome, especially when you have gigs and gigs of daily logs.
The fact that JaM has multiple backends is also helpful because you can configure it to send only a subset of events to one backend while logging all events to another.
For instance, think about emails to alert about issues. Many applications trigger a lot of notices you'd like to solve but are not critical for the application's operation. You therefore might not wish to get them by email, since there will be a lot of them and no immediate action will be taken to fix them.
You can configure JaM to send only what you consider to be critical errors by mail for an immediate response, and log the rest to ElasticSearch or any other backend for later analysis and resolution.
If I had one, could I run JaM to monitor my Wordpress install?
Yes, and it would only take about five minutes to configure provided your WordPress instance runs on Linux or any other UNIX. JaM should be able to run on Windows as well, but I've never tested it since I use Linux almost exclusively (with the occasional dive into FreeBSD).
How did you get started in PHP? You've obviously done a lot of work in it. Is that out of habit, or is it your preferred language for web development?
I started using PHP heavily when I worked for Zend, where I wrote PHP code, helped maintain various PHP extensions, and was responsible for building the Zend Server deb and RPM packages.
At Kaltura, we also make heavy use of PHP, as most of our core is written in it. So, I do get to write quite a lot of PHP code.
I think PHP certainly has some advantages where it comes to web development, one of which is that you can produce results very quickly.
With PHP 7, the performance boost is quite substantial, which traditionally was one of the problems with using PHP, so that's good news as well.
Incidentally, at Kaltura, we recently ported our Core to PHP 7.
Of course, it is very important to take the time to properly know it.
I think one of the pitfalls of being easy to develop in, is that you can start producing code that seems very functional without truly understanding much, and some beginners are clearly tempted to do so.
Like any other language, it's only as good as the developer using it. But all in all, I'd say for rapid web development, it is one my favorites, along with Ruby, which is also very nice and flexible.
JaM is very extensible. Are you actively looking for contributors? Do you have a to-do list filled with tasks needing to be done? Or is JaM in a good place right now?
I am a huge believer in FOSS, both from a technical and an ideological standpoint. Therefore, I am always happy to get contributions in the form of code, documentation, and bug reports.
It was very important to me that the project was open. Luckily, at Kaltura, most of our projects are. In fact, part of my job is to collaborate with community developers and users to integrate their contributions into our code base.
JaM is in a pretty good place, but it's very extensible by design and I would love to see people contributing additional backends. I also started porting it to PHP 7 and would love to have others join the effort.
You do a lot with open source. Why is open source important?
This is a topic I can spend hours talking about. It's a bit hard to summarize in just a few lines, but I'll try.
From a technical standpoint, being open makes any project better because no company can compete with the amount of resources the community has to offer as far as testing different use cases, security auditing, bug reporting, and fixing.
From an ideological perspective, open source means the users don't get a car with the hood welded shut. Even if a certain user is not technical at all and cannot understand the code themselves, being open means he or she can easily find someone to help them, more often than not, completely for free.
It means users do not get locked in, ending up with something they no longer want and cannot change. That's a basic freedom every end user should have.
From a social standpoint: FOSS is a great way to meet like-minded people and potential friends. I was and still am a very proud nerd, and I envy kids today for being able to find friends via their FOSS activities.
FOSS is also very good for one's career. While the code you write belongs to the company you work for, the credit for your work, belongs to you, and you can use it as reference throughout your career. Developers who work on FOSS software also tend to write better code because they know the world will be looking!
Do you run Linux on your desktop/laptop? If so, what distribution? What's your desktop or window manager?
I've run exclusively on Linux since the age of 15. I started with Red Hat 5 back in 1997.
My favorite desktop environment is LXDE. I like it because it's full-featured enough for all I need, and yet quite lightweight and has in fact never crushed on any of my machines, which is a rather important feature.
My desktop computer runs Debian, but I also have chroots and LXCs with CentOS.
Comments are closed.