This tutorial is the first in a two-part series on Spring
Integration. In this series we're going to build out a lead management
system based on a message bus that we implement using Spring
Integration. Our first tutorial will begin with a brief overview of
Spring Integration and also just a bit about the lead management domain.
After that we'll build our message bus.
I've used Maven profiles to isolate the dependencies
you’ll need if you’re running Java 5.
The tutorials assume that you're
comfortable with JEE, the core Spring framework and Maven 2. Also,
Eclipse users may find the m2eclipse plug-in helpful.
To complete the tutorial you'll need an IMAP account, and you'll also need access to an SMTP server.
Let's begin with an overview of Spring Integration.
A bird's eye view of Spring Integration
Spring
Integration is a framework for implementing a dynamically configurable
service integration tier.
The point of this tier is to orchestrate
independent services into meaningful business solutions in a
loosely-coupled fashion, which makes it easy to rearrange things in the
face of changing business needs.
The service integration tier sits just
above the service tier as shown in figure 1.
Following the book
Enterprise Integration Patterns by Gregor Hohpe and Bobby Woolf
(Addison-Wesley), Spring Integration adopts the well-known pipes and
filters architectural style as its approach to building the service
integration layer.
Abstractly, filters are information-processing units
(any type of processing—doesn’t have to be information filtering per
se),
and pipes are the conduits between filters.
In the context of
integration, the network we’re building is a messaging infrastructure—a
so-called message bus—and the pipes and filters and called message
channels and message endpoints, respectively.
The network carries
messages from one endpoint to another via channels, and the message is
validated, routed, split, aggregated, resequenced, reformatted,
transformed and so forth as the different endpoints process it.
Figure 1. The service integration tier orchestrates the services below it.
That
should give you enough technical context to work through the tutorial.
Let’s talk about the problem domain for our sample integration, which is
enrollment lead management in an online university setting.
Lead management overview
In
many industries, such as the mortgage industry and for-profit
education, one important component of customer relationship management
(CRM) is managing sales leads.
This is a fertile area for enterprise
integration because there are typically multiple systems that need to
play nicely together in order to pull the whole thing off.
Examples
include
- front-end marketing/lead generation websites,
- external lead
vendor systems,
- intake channels for submitted leads,
- lead databases,
- e-mail systems (e.g., to accept leads, to send confirmation e-mails),
- lead qualification systems,
- sales systems and potentially others.
This
tutorial and the next use Spring Integration to integrate several of
systems of the kind just mentioned into an overall lead management
capability for a hypothetical online university. Specifically we’ll
integrate the following:
• a CRM system that allows campus
and call center staff to create leads directly, as they might do for
walk-in or phone-in leads
• a Request For Information (RFI) form on a lead generation ("lead gen") marketing website
• a legacy e-mail based RFI channel
• an external CRM that the international enrollment staff uses to process international leads
• confirmation e-mails
Figure
2 shows what it will look like when we’re done with both tutorials. For
now focus on the big picture rather than the details.
Figure 2. This is the lead management system we'll build.
For
this first tutorial we're simply going to establish the base staff
interface, the (dummy) backend service that saves leads to a database,
and confirmation e-mails.
The second tutorial will deal with lead
routing, web-based RFIs and e-mail-based RFIs.
Let's dive in. We’ll begin with the basic lead creation page in the CRM and expand out from there.
Building the core components
[You can download the source code for this section of the tutorial
here]
We’re
going to start by creating a lead creation HTML form for campus and
call center staff. That way, if walk-in or phone-in leads express an
interest, we can get them into the system. This is something that might
appear as a part of a lead management module in a CRM system, as shown
in figure 3.
Figure 3. We'll build our lead management module with integration in mind from the beginning.
Because
we’re interested in the integration rather than the actual app
features, we’re not really going to save the lead to the database.
Instead we’ll just call a createLead() method against a local
LeadService bean and leave it at that. But we will use Spring
Integration to move the lead from the form to the service bean.
Our first stop will be the domain model.
Create the domain model
We’ll need a domain object for
leads, so listing 1 shows the one we’ll use. It’s not an
industrial-strength representation, but it will do for the purposes of
the tutorial.
Listing 1. Lead.java, a basic domain object for leads.
package crm.model;
... other imports ...
public class Lead {
private static DateFormat dateFormat = new SimpleDateFormat();
private String firstName;
private String middleInitial;
private String lastName;
private String address1;
private String address2;
... other fields ...
public Lead() { }
public String getFirstName() { return firstName; }
public void setFirstName(String firstName) {
this.firstName = firstName;
}
... other getters and setters, and a toString() method ...
}
There is nothing special happening here at all.
So far the Lead class is
just a bunch of getters and setters. You can see the full code listing
in the download.
If you thought that was underwhelming, just wait until you see the LeadServiceImpl service bean in listing 2.
Listing 2. LeadServiceImpl.java, a dummy service bean.
package crm.service;
import java.util.logging.Logger;
import org.springframework.stereotype.Service;
import crm.model.Lead;
@Service("leadService")
public class LeadServiceImpl implements LeadService {
private static Logger log = Logger.getLogger("global");
public void createLead(Lead lead) {
log.info("Creating lead: " + lead);
}
}
This is just a dummy bean. In real life we’d save the lead to a
database. The bean implements a basic LeadService interface that we've
suppressed here, but it's available in the code download.
Now that we have our domain model, let’s use Spring Integration to create a service integration tier above it.
Create the service integration tier
If
you look back at figure 3, you’ll see that the CRM app pushes lead data
to the service bean by way of a channel called newLeadChannel.
While
it’s possible for the CRM app to push messages onto the channel
directly, it’s generally more desirable to keep the systems you’re
integrating decoupled from the underlying messaging infrastructure, such
as channels. That allows you to configure service orchestrations
dynamically instead of having to go into the code.
Spring
Integration supports the Gateway pattern (described in the
aforementioned Enterprise Integration Patterns book), which allows an
application to push messages onto the message bus without knowing
anything about the messaging infrastructure. Listing 3 shows how we do
this.
Listing 3. LeadGateway.java, a gateway offering access to the messaging system.
package crm.integration.gateways;
import org.springframework.integration.annotation.Gateway;
import crm.model.Lead;
public interface LeadGateway {
@Gateway(requestChannel = "newLeadChannel")
void createLead(Lead lead);
}
We are of course using the Spring Integration @Gateway annotation to map
the method call to the newLeadChannel, but gateway clients don’t know
that. Spring Integration will use this interface to create a dynamic
proxy that accepts a Lead instance, wraps it with an
org.springframework.integration.core.Message, and then pushes the
Message onto the newLeadChannel.
The Lead instance is the Message body,
or payload, and Spring Integration wraps the Lead because only Messages
are allowed on the bus.
We need to wire up our message bus. Figure 4 shows how to do that with an application context configuration file.
Listing 4. /WEB-INF/applicationContext-integration.xml message bus definition.
<?xml version="1.0" encoding="UTF-8"?>
<beans:beans xmlns="http://www.springframework.org/schema/integration"
xmlns:beans="http://www.springframework.org/schema/beans"
xmlns:p="http://www.springframework.org/schema/p"
xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
xsi:schemaLocation="http://www.springframework.org/schema/beans
http://www.springframework.org/schema/beans/spring-beans-2.5.xsd
http://www.springframework.org/schema/integration
http://www.springframework.org/schema/integration/spring-integration-1.0.xsd">
<gateway id="leadGateway"
service-interface="crm.integration.gateways.LeadGateway" />
<publish-subscribe-channel id="newLeadChannel" />
<service-activator
input-channel="newLeadChannel"
ref="leadService"
method="createLead" />
</beans:beans>
The first thing to notice here is that we've made the Spring
Integration namespace our default namespace instead of the standard
beans namespace.
The reason is that we're using this configuration file
strictly for Spring Integration configuration, so we can save some
keystrokes by selecting the appropriate namespace.
This works pretty
nicely for some of the other Spring projects as well, such as Spring
Batch and Spring Security.
In this configuration we've created the
three messaging components that we saw in figure 3.
First, we have an
incoming lead gateway to allow applications to push leads onto the bus.
We simply reference the interface from listing 3;
Spring Integration
takes care of the dynamic proxy. Next we create a publish/subscribe
("pub-sub") channel called newLeadChannel.
This is the channel that the
@Gateway annotation referenced in listing 3.
A pub-sub channel can
publish a message to multiple endpoints simultaneously.
For now we have
only one subscriber—a service activator—but we already know we're going
to have others, so we may as well make this a pub-sub channel.
The
service activator is an endpoint that allows us to bring our
LeadServiceImpl service bean onto the bus.
We're injecting the
newLeadChannel into the input end of the service activator.
When a
message appears on the newLeadChannel, the service activator will pass
its Lead payload to the leadService bean's createLead() method.
Stepping
back, we've almost implemented the design described by figure 3. The
only part that remains is the lead creation frontend, which we'll
address right now.
Create the web tier
Our user interface for creating new leads
will be a web-based form that we implement using Spring Web MVC. The
idea is that enrollment staff at campuses or call centers might use such
an interface to handle walk-in or phone-in traffic. Listing 5 shows our
simple @Controller.
Listing 5. LeadController.java, a @Controller to allow staff to create leads
package crm.web;
import java.util.Date;
import org.springframework.beans.factory.annotation.Autowired;
import org.springframework.stereotype.Controller;
import org.springframework.ui.Model;
import org.springframework.web.bind.annotation.RequestMapping;
import org.springframework.web.bind.annotation.RequestMethod;
import crm.integration.gateways.LeadGateway;
import crm.model.Country;
import crm.model.Lead;
@Controller
public class LeadController {
@Autowired
private LeadGateway leadGateway;
@RequestMapping(value = "/lead/form.html", method = RequestMethod.GET)
public void getForm(Model model) {
model.addAttribute(Country.getCountries());
model.addAttribute(new Lead());
}
@RequestMapping(value = "/lead/form.html", method = RequestMethod.POST)
public String postForm(Lead lead) {
lead.setDateCreated(new Date());
leadGateway.createLead(lead);
return "redirect:form.html?created=true";
}
}
This isn't an industrial-strength controller as it doesn't do HTTP
parameter whitelisting (for example, via an @InitBinder method) and form
validation, both of which you would expect from a real implementation.
But the main pieces from a Spring Integration perspective are here.
We're autowiring the gateway into the @Controller, and we have methods
for serving up the empty form and for processing the submitted form.
The
getForm() method references a Countries class that we've suppressed
(it's in the code download);
it just puts a list of countries on the
model so the form can present a Country field to the staff member.
The
postForm() method invokes the createLead() method on the gateway.
This
will pass the Lead to the dynamic proxy LeadGateway implementation,
which in turn will wrap the Lead with a Message and then place the
Message on the newLeadChannel.
There are a few other configuration
files you will need to put in place, including web.xml,
main-servlet.xml and applicationContext.xml.
There's also a JSP for the
web form. As none of these relates directly to Spring Integration, we
won't treat them here. Please see the code download for details.
With that, we've established a baseline system.
To try it out, run
mvn jetty:run
against crm/pom.xml and point your browser at
http://localhost:8080/crm/main/lead/form.html
You
should see a very basic-looking web form for entering lead information.
Enter some user information (it doesn't matter what you enter—recall
that we don't have any form validation) and press Submit.
The console
should report that LeadServiceImpl.createLead() created a lead.
Congratulations!
Even though we now have a working system, it
isn't very interesting. From here on out (this tutorial and the next)
we'll be adding some common features to make the lead management system
more capable.
Our first addition will be confirmation e-mails;
Adding confirmation e-mails
After
an enrollment advisor (or some other staff member) creates a lead in
the system, we want to send the lead an e-mail letting him know that
that's happened. Actually—and this is a critical point—we really don't
care how the lead was created. Anytime a lead appears on the
newLeadChannel, we want to fire off a confirmation e-mail. I'm making
the distinction because it points to an important aspect of the message
bus: it allows us to control lead processing code centrally instead of
having to chase it down in a bunch of different places. Right now
there's only one way to create leads, but figure 2 revealed that we'll
be adding others. No matter how many we add, they'll all result in
sending a confirmation e-mail out to the lead.
Figure 4 shows the new bit of plumbing we're going to add to our message bus.
Figure 4. Send a confirmation e-mail when creating a lead.
To do this, we're going to need to make a few changes to the configuration and code.
POM changes
First we need to update the POM. Here's a summary of the changes; see the code download for details:
• Add a JavaMail dependency to the Jetty plug-in.
• Add an org.springframework.context.support dependency.
• Add a spring-integration-mail dependency.
• Set the mail.version property.
These changes will allow us to use JavaMail.
Expose JavaMail sessions through JNDI
We'll
also need to add a /WEB-INF/jetty-env.xml configuration to make our
JavaMail sessions available via JNDI.
Once again, see the code download
for details.
it's included a /WEB-INF/jetty-env.xml.sample configuration
for your convenience.
As mentioned previously, you'll need access to an
SMTP server.
Besides creating jetty-env.xml, we'll need to update
applicationContext.xml.
Listing 6 shows the changes we need so we can
use JavaMail and SMTP.
Listing 6. /WEB-INF/applicationContext.xml changes supporting JavaMail and SMTP
<?xml version="1.0" encoding="UTF-8"?>
<beans xmlns="http://www.springframework.org/schema/beans"
xmlns:context="http://www.springframework.org/schema/context"
xmlns:jee="http://www.springframework.org/schema/jee"
xmlns:p="http://www.springframework.org/schema/p"
xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
xsi:schemaLocation="http://www.springframework.org/schema/beans
http://www.springframework.org/schema/beans/spring-beans-2.5.xsd
http://www.springframework.org/schema/context
http://www.springframework.org/schema/context/spring-context-2.5.xsd
http://www.springframework.org/schema/jee
http://www.springframework.org/schema/jee/spring-jee-2.5.xsd">
<jee:jndi-lookup id="mailSession"
jndi-name="mail/Session" resource-ref="true" />
<bean id="mailSender"
class="org.springframework.mail.javamail.JavaMailSenderImpl"
p:session-ref="mailSession" />
<context:component-scan base-package="crm.service" />
</beans>
The changes expose JavaMail sessions as a JNDI resource.
We've
declared the jee namespace and its schema location, configured the JNDI
lookup, and created a JavaMailSenderImpl bean that we'll use for sending
mail.
We won't need any domain model changes to generate
confirmation e-mails. We will however need to create a bean to back our
new transformer endpoint.
Service integration tier changes
First,
recall from figure 4 that the newLeadChannel feeds into a
LeadToEmailTransformer endpoint. This endpoint takes a lead as an input
and generates a confirmation e-mail as an output, and the e-mail gets
pipes out to an SMTP transport.
In general, transformers transform given
inputs into desired outputs. No surprises there.
Figure 4 is
slightly misleading since it's actually the POJO itself that we're going
to call LeadToEmailTransformer; the endpoint is really just a bean
adapter that the messaging infrastructure provides so we can place the
POJO on the message bus.
Listing 7 presents the LeadToEmailTransformer
POJO.
package crm.integration.transformers;
import java.util.Date;
import java.util.logging.Logger;
import org.springframework.integration.annotation.Transformer;
import org.springframework.mail.MailMessage;
import org.springframework.mail.SimpleMailMessage;
import crm.model.Lead;
public class LeadToEmailTransformer {
private static Logger log = Logger.getLogger("global");
private String confFrom;
private String confSubj;
private String confText;
... getters and setters for the fields ...
@Transformer
public MailMessage transform(Lead lead) {
log.info("Transforming lead to confirmation e-mail: " + lead);
String leadFullName = lead.getFullName();
String leadEmail = lead.getEmail();
MailMessage msg = new SimpleMailMessage();
msg.setTo(leadFullName == null ?
leadEmail : leadFullName + " <" + leadEmail + ">");
msg.setFrom(confFrom);
msg.setSubject(confSubj);
msg.setSentDate(new Date());
msg.setText(confText);
log.info("Transformed lead to confirmation e-mail: " + msg);
return msg;
}
}
Again, LeadToEmailTransformer is a POJO, so we use the
@Transformer
annotation to select the method that's performing the transformation.
We
use a Lead for the input and a MailMessage for the output, and perform a
simple transformation in between.
When defining backing beans for
the various Spring Integration filters, it's possible to specify a
Message as an input or an output.
That is, if we want to deal with the
messages themselves rather than their payloads, we can do that. (Don't
confuse the MailMessage in listing 7 with a Spring Integration message;
MailMessage represents an e-mail message, not a message bus message.) We
might do that in cases where we want to read or manipulate message
headers. In this tutorial we don't need to do that, so our backing beans
just deal with payloads.
Now we'll need to build out our message
bus so that it looks like figure 4. We do this by updating
applicationContext-integration.xml as shown in listing 8.
Listing 8. /WEB-INF/applicationContext-integration.xml updates to support confirmation e-mails
<?xml version="1.0" encoding="UTF-8"?>
<beans:beans xmlns="http://www.springframework.org/schema/integration"
xmlns:mail="http://www.springframework.org/schema/integration/mail"
xmlns:beans="http://www.springframework.org/schema/beans"
xmlns:context="http://www.springframework.org/schema/context"
xmlns:p="http://www.springframework.org/schema/p"
xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
xsi:schemaLocation="http://www.springframework.org/schema/integration/mail
http://www.springframework.org/schema/integration/mail/spring-integration-mail-1.0.xsd
http://www.springframework.org/schema/beans
http://www.springframework.org/schema/beans/spring-beans-2.5.xsd
http://www.springframework.org/schema/context
http://www.springframework.org/schema/context/spring-context-2.5.xsd
http://www.springframework.org/schema/integration
http://www.springframework.org/schema/integration/spring-integration-1.0.xsd">
<context:property-placeholder
location="classpath:applicationContext.properties" />
<gateway id="leadGateway"
service-interface="crm.integration.gateways.LeadGateway" />
<publish-subscribe-channel id="newLeadChannel" />
<service-activator
input-channel="newLeadChannel"
ref="leadService"
method="createLead" />
<transformer input-channel="newLeadChannel" output-channel="confEmailChannel">
<beans:bean class="crm.integration.transformers.LeadToEmailTransformer">
<beans:property name="confFrom" value="${conf.email.from}" />
<beans:property name="confSubject" value="${conf.email.subject}" />
<beans:property name="confText" value="${conf.email.text}" />
</beans:bean>
</transformer>
<channel id="confEmailChannel" />
<mail:outbound-channel-adapter
channel="confEmailChannel"
mail-sender="mailSender" />
</beans:beans>
The property-placeholder configuration loads the various ${...}
properties from a properties file; see
/crm/src/main/resources/applicationContext.properties in the code
download.
You don't have to change anything in the properties file.
The
transformer configuration brings the LeadToEmailTransformer bean into
the picture so it can transform Leads that appear on the newLeadChannel
into MailMessages that it puts on the confEmailChannel.
As a side note,
the p namespace way of specifying bean properties doesn't seem to work
here (I assume it's a bug:
http://jira.springframework.org/browse/SPR-5990), so I just did it the
more verbose way.
The channel definition defines a point-to-point
channel rather than a pub-sub channel. That means that only one endpoint
can pull messages from the channel.
Finally we have an
outbound-channel-adapter that grabs MailMessages from the
confEmailChannel and then sends them using the referenced mailSender,
which we defined in listing 6.
That's it for this section. We should have working confirmation e-mails. Restart your Jetty instance and go again to
http://localhost:8080/crm/main/lead/form.html
Fill
it out and provide your real e-mail address in the e-mail field. A few
moments after submitting the form you should receive a confirmation
e-mail. If you don't see it, you might check your SMTP configuration in
jetty-env.xml, or else check your spam folder.
Summary
In
this tutorial we've taken our first steps toward developing an
integrated lead management system. Though the current bus configuration
is simple, we've already seen some key Spring Integration features,
including
• support for the Gateway pattern, allowing us to connect apps to the message bus without knowing about messages • point-to-point and pub-sub channels • service activators to allow us to place service beans on the bus • message transformers • outbound SMTP channel adapters to allow us to send e-mailThe second tutorial
will continue elaborating what we've developed here, demonstrating the
use of several additional Spring Integration features, including • message routers (including content-based message routers) • outbound web service gateways for sending SOAP messages • inbound HTTP adapters for collecting HTML form data from external systems
• inbound e-mail channel adapters (we'll use IMAP IDLE, though POP
and IMAP are also possible) for processing incoming e-mails
