 Hi, I'm John Sabo. I'm with Rogue Wave Software and I'm going to be presenting on Apache Camel. Let's see. I'm apparently at the end of my presentation. Let me get back to the right spot here. Here we go. Now we got it. Okay. And making Apache Camel work for you. So, I'm an enterprise architect with the OSS support team at Rogue Wave Software. We support a big variety of open source products. I personally do a lot of work with Apache ActiveMQ, Apache Camel, and then also work with Tomcat, other web servers, that kind of thing. Anyways, Camel is an open source message oriented middleware language. It's developed by Apache. It's based on enterprise integration patterns. There's a book by Gregor Hope and Bobby Wolf called Enterprise Integration Patterns. It's a wonderful book. It has about 60 plus common patterns that you would commonly use enterprise integration projects. This includes things like, let's see, blanking right now, but lots of patterns. Disturbing messages across different processors, doing transformations, et cetera. We'll get into that later in the presentation. It defines these patterns in a vendor-neutral way. These are things you'd want to do commonly in any message-oriented environment in the enterprise. Camel aims to provide a language that implements these patterns in a standardized way. You know they're implemented correctly. All that boilerplate is already there. It's very simple to use and gets you up and running very quickly. And it's efficient at it, too. It's modeled after the UNIX pipeline. So everything has an input and an output. It's all based around messaging. Camel's a huge topic. There's a lot of inputs and outputs that you can use with it. A lot of different processors available. You can even code your own. So we're just going to scratch the surface here and just barely kind of do a demo, talk about what it can do, and kind of get you going to where you can start investigating it yourself. How does it work? Camel uses this concept of processor and message objects in a group called an exchange. So how that works? There's an in message. There's a processor and then an out message. The processor performs whatever transformations, filters, whatever you want to do on that message as it comes in and then outputs it into an out message. You know, same semantics. So you have an in message, a processor, and an out message. Because this is, you know, it's consistent, you can chain these together as much as you want. You can generate a chain of processors here to, you know, transform message, grab content from another source to add to that message, decorate it. You can then, you know, distribute that message, perform filtering on it. Basically, whatever you need to do to transform this. Just like a Unix pipeline. So, you know, just like you'd pipe things through the Unix pipeline, you might, you know, tee something out to a file, etc. Camel does this to messages and for enterprise integration. The lab will actually be a content-based router kind of based on this. The out message from your previous exchange becomes the in message for your next exchange or you, you know, shoot it out to another file, a HP endpoint, JSON, you know, anything you want. It can go to another message bus. So these change, the chain together pretty consistently here. You have a pipeline. It's like Lego blocks. The magical part here is that the processor can be anything that you want it to be. You can create your own processors to do whatever you need to do in your business. If you have some sort of custom processing you need to do, you can do that. You can have, there's an FTP processor that can either grab from an FTP site or you can initiate that session to pull something. You can also output to FTP. Same way. SMTP. You can grab, you know, mail messages or you can send mail messages. These are all things you can do. So basically it's input from anything, output to anything else. It can be triggered or it can be on a timer, however you need to do this. It can be triggered by another message coming in and then triggering it to, you know, triggering the processor to pull from another source or to combine it with another message. There's a lot of built-in patterns mentioned, you know, some of these here. Load balancing. You can load balance a certain type of message across any number of outputs. You can multicast. So you can, you know, multicast out to several different end points that you want to send to. You can create data sets with aggregation. And there's a ton of these components that are just built in. Like I said, there's 60 plus integration patterns. There's probably a hundred plus components. And let me, let me actually look at and bring this up here just so you can see. This is the list on Camel's site of all the components available just out of the box. So we can look through here. You can, you know, add an integration. AWS SimpleDB email service. You can do, let's see, Cassandra, CouchDB, CXF for web services. You can do, losing my cursor here, Docker. You can communicate back and forth through your Docker instances, Dropbox, Elasticsearch, etcd for configuration. Process spring events. Is Git hooks. Let's see. What else is in here? You can do hip chat. You can do integration with chat as well. So if you want to process, you know, if a message comes in, you want to throw it onto your chat client so that you can display it in chat where it's relevant. You can do that for alerting or things like that based on certain messages. IMAP, IRC, JBPM, JMX. So you can monitor certain JMX entities using this. JDBC. So you can pull from and push into databases. Let's see. Kubernetes. You can interact with that. Kafka. So you can use Kafka message brokers. ActiveMQ is in there too. I mean, so you get the idea. There's like inputs and outputs of everything. PDF, OpenShift, Pop, Quartz, RabbitMQ, REST. Anyways, way too much. In any case, so there's a ton of components that are built into Camel. This is a cropped picture. It doesn't actually, it stops at Scala here. But yeah, so inputs and outputs from almost anything out of the box. And you can always code your own if there's something that isn't in there already for some reason. So Camel is pretty awesome. Input and output transformation, message transformation from any input and output. It provides a lot of benefits for enterprise messaging applications. There's over 100 integration components and a framework that's pretty easy to create your own components if the out of the box stuff doesn't do exactly what you want. It's very configurable so you can do almost anything there. You can do, Camel does have visual debugging now. Similar to JPPM or BPEL. That's a newer feature. It wasn't there at the beginning. Camel cuts down on boilerplate code because you can express very concisely in its spring-based integration components, you know, different transformations. So if you have simple transformations and processes and chains that you want to implement, there's a spring-based DSL that is extremely compact for expressing most of these patterns that you'd want to implement. It is FOS. It's under the Apache 2 license. Obviously, we're at ApacheCon. So it is an Apache project. They continue to work on the underlying spring libraries so spring advances, Camel's expressiveness increases as well. It's pretty easy to understand and has a low learning curve to implement. I'll show a few common EIPs, enterprise integration patterns that Camel includes out of the box here. Let's see. Go here. So first one, dynamic router. You might want to route different messages to different endpoints based on message criteria. So this is actually what I'm going to do the lab on. And say you have two different types of messages coming in. One needs to be processed by one processor, another by another one. So this example, you have a widget and widgets and gadgets and you keep track of the inventory and those in two different systems. You have all your inventory coming into this from the order. This content-based router is splitting those orders up automatically into widgets and gadgets. So pretty straightforward there. It's a common enterprise integration pattern. Content and richer. So in this example, you have a basic message. It's pretty bare bones. Say for that order, for the last one, maybe it just has a part number and a quantity. But before you deploy it to your user, you need to pull where the image resource for it is and the name of it. So this in richer will pull that inventory number, ping a database to find out what the long description of the item is. Maybe some additional resources attached to that. Attach it to that message. The message coming out of that has been enriched with that extra content to that point. Let's see. Recipient list. So recipient list broadcast message comes in. You broadcast it to end channels based on the message criteria. If you broadcast it to all, this would be multicast. So that's the kind of degenerate case here of broadcasting to all as multicast. But say you might want to select only certain end points or channels that this message gets routed to. So recipient list will do that. Here's another really powerful one is scatter gather. So this one, normally this is something you end up with a lot of code to do. Out of the box is pretty simple though. You take in, say, a quote request. You need to pull information from multiple vendors. This would be an example of like a travel booking website where you're comparing the prices for a certain flight route from multiple vendors or hotel prices or something like that. So what you want to do is you want to take this one request of this hotel this night and I'm going to split out to three different vendors and get the pricing from those three vendors. I'm going to grab those quotes. I'm going to then take it in and put it into an aggregator that then does a max on those quotes that I'm getting back. And then from that I get a best quote and then that's what I end up presenting to the user. And this is extremely simple to do with Camel. This is a few lines. You specify message, what you want to filter out of it, where you're querying based on that filtered data. Those quotes that come in, what you want to filter out of those, you aggregate, you find the max and then you have an output of your end result here. So let's see. Let's go ahead and dive in. Then coding in Camel. So how do you do, how do you implement this? Let's see. How are we doing in time here? Okay. Camel routes, there's a lot of different ways you can express Camel routes. It's fairly flexible based on how you want to work. They can be coded in Java pojos. So if you like working in code and that's how you like to implement things, it's pretty simple to do this. This is an example of a Camel route right here. It's extremely concise and extremely powerful. What we're doing here is we're taking messages from the foo queue on our ActiveMQ broker. We are consuming from that foo queue. We're applying a filter. With that, in that filter, we're looking for an X query. So we're filtering on foo. So we're immediately, we're getting XML messages. We're filtering them based on an X query, based on the messages that match that. We are then forwarding those messages on that match that X query filter into a separate ActiveMQ bar on our ActiveMQ instance. This is, it's like a one liner in Camel. If you're coding this manually, it'd probably be a little bit more involved and probably wouldn't be as concise and understandable. You can also do this very simply and if you're prototyping and if you're just trying to play around with this and even in production, this is a great way to do things because it's so easy. You can use the Spring DSL built into Camel to implement these different integration patterns and message crossing in straight up XML files, just when configuration and update these as you need to. So here is another example of a Camel route. So we have our Camel route here, Spring, we're calling it Timer Client. So what we're going to do here is in a couple lines here, so we have the same structure we have from our, just like anything else in Camel, it's input, processor, output. So here's from, we're instantiating in this one line here, we're creating a timer and this timer is going to repeat 10,000 times and it's going to repeat once every 10 seconds. So in this one line we are creating, we're instantiating Quartz, we're telling it every 10 seconds to trigger. Every 10 seconds we are triggering the production out of an active MQ message. We're calling this AMQ load timer and we're doing this 10,000 times in this one line. So we're generating load is basically what we're doing. So from here, we're setting the body of these messages to, this is a test JMS message. So we're just basically generating a bunch of messages to this active MQ queue with the content, this is a test JMS message. We are then sending those messages to an output of our active MQ test active MQ route in queue. That's our ID there is timer client in queue producer, that can be useful when you're then looking at how this is acting. This is a great way using a timer like this when you're testing out routes. If you're even, even if you're not using Camel, if you have a messaging system and you need to generate some test load, using a timer and Camel like this is an extremely easy way to do that. It takes one line, there's no compilation step. You stick this in the, in your Camel route configuration and start it and there you have 10,000 messages going into your message queue for test purposes. So it's pretty simple there. Now, let's see this Java pojo up there. All this is implementing, you're using the route builder class for that. You cannot, the one drawback to the Java pojos is you can't render these routes graphically in hot.io, which is extremely useful when you're debugging routes in transformations with this Spring DSL. As you're mocking these things up, you can actually see the messages propagate through your chain of transformations and debug them that way. So it's a very useful tool. Once you get a production or want to speed things up or do some more custom things, you can always do the pojos as well. Let's see. And we'll have a demo on that as well. Let's see. This, yeah. So this is our example Spring DSL Camel route. I just talked through it back there. We send 10,000 messages every 10 seconds per message and we're sending it into an active MQ queue. In hot.io, when we're, when we want to view this graphically, this is what it ends up looking like. We'll see a box here for our from. We are source is this timer source and we're calling AMQ load timer. The message contents is this is a test. JMS message. And then it shows us two and our sync here is our active MQ queue test active MQ route in queue. And that counter up in the top there, as you can see, 1510. That's just the number of messages that have gone through at this point in the GUI view. Let's see. The developers on the Camel project overlap a lot with active MQ. So that's part of the reason why a lot of the demos are using active MQ. It's actually compatible with any JMS provider though. Active MQ is just the one that they tend to use for demos. And as I showed you, it's compatible with other message buses and things as well. Okay. Message anatomy. So Camel is based around messages. It's very similar to the anatomy of a JMS message. That's kind of how it represents things internally. It's a normalized message router. So that's not very surprising. The normalized part is because the exchange patterns all operate with the same structure. There's an in message, there's an out message, and a processor. The message part is the fact that the message is of the payload. So messages can be anything. Camel doesn't even have to understand the contents of a message as long as you're not, the processors might understand the contents of a message, but fundamentally Camel doesn't care what's in the message. It's a payload. So it doesn't matter if it's something that Camel doesn't understand. You can still process it. And here's kind of what the message structure looks like. There's a message ID. There's headers, properties, payload, very similar to JMS messages. Embedded processors. So Camel contains a bunch of embedded script languages. We saw the spring DSL, but it contains a number of embedded script languages that you can use to create your own processors without even having to go outside of Camel. The simplest one, ironically enough, is called Simple. It's a very simple query language. You can use it to introspect on message headers, payloads, properties. It's very useful for routing messages based on their content. It's limited to looking up contents of those different message pieces. It can do some amount of filtering. It can do operations, but only single operations. It's not a full-fledged expression language. It is limited. But it makes it extremely simple to read, quick to implement if you're just trying to filter on something. Simple is great. A little bit more complex, XPath. So you saw on the Pojo Java example, there was a XPath query language option. You can use XPath to reverse XML structures inside your messages if they're XML messages as much as you want. So full featured XPath. You can filter based on anything you normally filter on an XPath, make decisions based on an XPath query. JSON path, same thing, but for JSON structured messages. So if your messages contain JSON, go, you know, have fun with that. You can filter based on that too. Groovy as well. So Groovy is also another scripting language available for Camel, and you can do pretty much anything you want with that because it's groovy and it's fully featured, and you can do pretty much anything you need to with that. You can also use JavaScript. So if you want to run JavaScript inside of your camel, you can do that. You can write your own processors in JavaScript and not even have a compile step. And, you know, you're good to go. So I'm going to do a quick demo here of a content-based router. We'll use the simple query language to introspect a JMS header in a message and route it to a destination based on the contents of that header. We'll use Camel deployed directly into ActiveMQ. ActiveMQ actually is bundled with the camel core distribution. It's already embedded in the ActiveMQ distribution that you download. So we're going to use a clean ActiveMQ install from scratch. And let's get that up here. So I'm going to extract my clean ActiveMQ distribution here, and then we will get to work on it. Okay, so this is completely clean. ActiveMQ 514.5 is the latest version. And if you look in example configurations, there is a camel XML here. I'm going to copy that into the conf directory. And then we have that already. Let's see. Let's get this up here. And okay, so here's our ActiveMQ.XML. I'm going to just add in a reference to camel. And so now we've included our camel config file here. Now I'm going to open up the camel config here. And since I've already done this, I'm going to just paste it in because I think we're running a little low on time. Here we go. Okay, so what I'm going to do here is let's see. I'm going to change the broker URL. So the broker I'm connecting to, I'm going to connect to the local host. And then over here, there is an example route in here already. It just sends messages that are in a QA on our ActiveMQ instance to QB. So instead, I'm going to do something a little fancier with a content-based router. And with this content-based router, I'm calling it CBRQ. It's content-based router. It is going to filter. There's a choice structure in the Spring DSL for camel. This choice structure is kind of your if-then-else for camel. You have win clauses. I'm using the simple query language here that we talked about. I'm filtering on my JMS type header, so the JMS type for the message. And I'm looking to see if it's type 1, type 2, or fall through and otherwise. Then, depending on which type it is, I'm going to, I have another two statement in here. And this two statement is either going to send that message along to type 1Q, type 2Q, or exception Q. So let's go ahead and save this here. And then I will start up ActiveMQ. And let's see here. So if you look here, this is a plain vanilla ActiveMQ installation I've done nothing to other than editing those two files. And of course, my syntax is off. So I will fix that here. That's what happens when you paste in looking over your shoulder. Let's see. So let's try that again. So we're waiting here for ActiveMQ to start up. camel, if you recall, is actually the core is built into the ActiveMQ distribution. So I didn't have to add anything else to this. It already has this, just had to make a camel config file. Okay, so starting up. And then if I look over here, let's see, wait for it to start up. Almost started. Deloquia, okay, getting there. It takes a little while to start up. And as soon as this starts up here, okay, now we have our ActiveMQ console. So what we can see here, this is just the default ActiveMQ console. It's running on port 8161 on my machine. Can open up my ActiveMQ broker. I see because camel's here and I've set up those queues, it already has a CVR queue. So as you can see, there's nothing in it right now. There are a couple of consumers. Those were based on those two processors that I implemented there in the when clause. So there's no messages in here right now. If I want to, I can send a message. I'm going to make my message type one here and I'm going to send it into my CVR queue. Okay, so went through. Now if I refresh here, I now have a type one queue and inside that queue, I have my message. So camel just routed that message for me and let's see here and that and actually I can see it in the console as well. So I sent that message in and it, let's see, I don't have debugging on, but yeah. So anyways, you can see that if I go into queues and I send here, I can do a type two message. That'll end up in my type two queue. If I have a message of some random type, that ends up in my exception queue. So that's pretty straightforward there. Let's see. How am I doing on time here? Should be good. Okay, let's go back to the slide deck here. Okay, camel deployment. Camel can be deployed in a bunch of different ways. It can be deployed as part of a Java web app using Spring or Pojo. Stand alone camel routes can be deployed using Maven and Camel. So you can, you can deploy with Maven. Spring boot can launch camel routes. It can be embedded in other Java applications. It can be embedded within Jetty, Tomcat, Jboss, Wildfly, whatever you want. ActiveMQ contains the camel core libraries like I mentioned. So you can launch and deploy camel routes right out of the standard ActiveMQ distribution. OSDI containers like CARAF or ServiceMEX, which includes ActiveMQ, Camel, and CXF can also launch camel routes. Major IDEs like Eclipse and Out in IntelliJ actually can allow you to execute the routes within the ID. There are comprehensive inline tools as of Camel 2.16 that you can autocomplete, list any parameter available on these different components. So here's an example of the file component. Here's autocomplete. How often you want to check that file location, what time unit, if you're logging level, scheduler, etc. Let's see. HotAO is probably the easiest and quickest way to prototype with Camel though. It lets you do visual debugging routes. I kind of showed you an example of that earlier. Tracing the message lifecycle and lets you diagram your development as well as monitoring. So when you have an active system running camel, you can run HotAO and see any of these routes we've created using the Spring DSL. This is an example here. So let's see. Let's see here. Yeah. So I will do a quick demo of that. Get this up and running. So I'm going to run a new terminal here. I already have HotAO in here. So I'm just going to run it Java.jar. Let's see. Head it in here. Let's check. Yeah. Why is that? Oh, there we go. So I'm starting up HotAO now. And this takes a little while. And as soon as it starts, it's going to auto launch a web console here. Let's see. So HotAO is great. If you haven't used it before, it lets you introspect on JMX. You can view your active MQ instances and Qs. And wonderfully, it lets you view your camel routes graphically. And as soon as it comes up here, okay, here we go. So it's still kind of coming up here, but it's a little bit slow. Okay. So here's HotAO. So as you can see in here, this is our local active MQ instance. It has all of our Qs that are in the instance right here. This is great for if you ever are running active MQ, you can debug what's going on. You can see, you know, it's much nicer looking than the active MQ built-in web console. It's a little fancier and stuff. See diagrams here of what is going on with your active MQ Qs. But what you can do in here is, if you go to the camel tab over here, it has a diagram of the route we just made in our XML. So what we can see here is, you know, we have our CBRQ input. We have a choice statement here. It tells us that three messages have already gone through. And you can even see the message statistics of how long they took to process about seven milliseconds here. And then the three different outputs here where, you know, we have choice or when clauses based on these three different simple filters. We're looking for type one, type two, or our otherwise fall through. And we can see here we have three messages go through the choice block and one ended up in each of the two blocks here. Now what's really neat here is I can debug this. So I can go right here and where's the debug here. There it is. Yeah, so start debugging. I'm going to add a breakpoint on my choice block here. So you see a little dot shows up there. I have a breakpoint. Let me send another message into my queue here, another type one message. And obviously this would work for much more complex chains as well. So sent one in. Now my debugger has stopped on my breakpoint. And I now see that this message and its name is kind of funny because I haven't really been doing anything nice with the names like I do in a real application. It has the message I just sent. Now if I want, I can step into it. So now I'm at the choice. And then actually there's only one step here. So ends up in two. So it just showed me that that message ended up in my first two clause filtered on type one. If I do another message here, it will do the same thing. Let's say type of something else. So it should end up in the exception queue. And right here, let's see, start. And okay, so here's my next message. So I've gotten four messages through this choice block. Here I can view the message down here. I'm going to go ahead and let it go through. And now I have two messages down here. So I can, I'm going to disable this breakpoint now. And so we're done debugging. Now what's really cool here additionally is there's also this trace function. I lost my cursor here. And it's not showing here because the monitor is a little too small. But I have trace right here. So I can start tracing. And what I can do, I can just send anything into here. And let's do type two here. Now, so I did that. Now the trace function here should show me the trace. It looks like I'm having some issues based on the screen size here. What that should be doing here, yeah, for some reason it's not working with the screen size. It wants a larger screen size. Normally what you'd be seeing here is a really cool trace of the messages in flight on each step of this route. So you'd see it come in to the CBRQ. You'd see the choice block. You'd see it hit the filter for type two. And then you'd see it end up in the type two queue. But it's not like in the screen size right now. So it's not showing that. Let's see. To continue on here, I think we're running low on time. Camel monitoring. Camel has a bunch of JMX instrumentation beans that can be looked at to gain more information on the route. As you could see in hot I.O., it tracks things, latency, maximum time, minimum time, average time on how your routes are doing. Any JMX-capable M&R prize monitoring solution is a good fit for monitoring this. So if you're in plenty of routes in Camel, you can monitor what your load is like. Here's an example just looking with JVVM looking into the different statistics available for each route. And actually I jumped the gun here. I went and did that hot I.O. and Camel demo there. I showed you setting break points, inspecting the message at that break point, and the trace function, which I guess you need a slightly larger resolution monitor to view that effectively. In conclusion, this is just like scraping the surface of Camel. We only used one processor and one message source. We're only using active MQ here. There's a bunch of powerful aggregation strategies that you can use to combine large, multi-threaded, multi-process data sets together. This is Camel running on one instance. If you had a truly large environment that you're trying to do this in, process large numbers of messages at volume, you might have more than one Camel instance running across your servers. This is just kind of debugging here. You might have a multi server farm of message Q servers that messages are being ingested into with a farm of Camel servers on your input and more Camel servers on your output, depending. But to get started, you can run just active MQ with this embedded Camel instance. It has all sorts of capabilities for message transformation using SXLT, velocity free marker, more advanced foreign conversion libraries like POI or PDF box. This is all kind of out of the box for Camel. You can make your own as well. Best thing is just start playing with it. It's really easy to play with. You saw the XML, the Spring DSL. It takes literally two minutes to get going with it. You just download active MQ, start playing with it, download Hadeo separately if you want to see things visually. The Camel community is pretty active and friendly. They have an IRC channel on FreeNode and, of course, the community page. That's Camel. Does anyone have any questions or anything about Camel, active MQ, anything else? Yes. So it's a little bit different just in terms of it. I haven't used spring integration as much. But it's meant to be a kind of a self-contained integration pattern library. So you can do input and output from anything. It's supposed to be infinitely flexible. You could use along with that, I'm sure. Do you have a specific use case you're looking for? Okay. Yes. Actually, I think I'd have to look. I don't remember if spring integration actually does anything with Camel. But I don't remember. It's been a while since I've touched it. Yes. Well, you can write pojos with Camel. Yes. Well, yes. So the nice thing about Camel, and it's kind of like the active MQ things, as you gain more load, you add more servers. You expand horizontally. Since everything is messages, it should be fairly easy to do that. That's why if Camel routes, it allows you to do things like load balance and things like that. Yeah. So what Hot.io is actually doing is it's just looking at the JMEX parameter is available to Camel. So Camel keeps track of your message, processing time, in-flight time, your averages, your maxes, all of that. Hot.io provides a very nice interface to view that graphically. Whereas oftentimes if you had that, you might have a framework that has a bunch of JMEX available to it. But it's relatively opaque. I mean, yes, you can go and view it. But it's not going to show all this stuff, hey, what's my route doing? And then just be able to trace through. You actually have to start looking through all the names and all that. Hot.io presents it graphically. So it's very nice for being able to drill down your routes and your processing chains and see where your bottlenecks are. And then you can optimize it better. As you kind of saw, as I was letting you bring up Hot.io again here. And yeah, so if you saw on this, like when I'm just moused over here, it's telling me what my mean min, max, and last processing time was. Now, the max and mean here and stuff is crazy high because I was tracing it. So I was doing some interesting stuff here. But it's very easy. You can connect this to any system. It's pretty simple. You don't want to run trace on production. But if you just want to start looking into your routes and your statistics and say like, hey, at this point in the route, my processing time is this. It's looking like this. I can start looking at that very quickly to drill down. You mentioned performance concerns and how am I going to scale. So this graphically is a very simple way to do that. Once you get more established in what processing you're doing, obviously you'd probably end up integrating that into your own JMX monitoring solutions, monitoring the routes that you normally run. But as you're prototyping and trying to determine your load, this is very useful. Any other questions? Let's see. I think we're good. Yeah.