Tag Archives: web-services

Remove null values from JSON object and ignore unavailable properties

The definition of web services using Spring is quite easy (see here for details), based on Jackson, the data marshalling/unmarshalling is totally hidden to the developer and it works perfectly.
But I encountered performance issues with the null values, I had to manage some classes with a lot of data, and they often contain empty values, resulting in a Json which was mainly composed of empty attributes (a lot) and it takes time to transfer it on the wire. To reduce the amount of data to transfer and then improve the response performance, it is possible to ask Jackson to include only non null values:

<bean id="jacksonObjectMapper" class="com.fasterxml.jackson.databind.ObjectMapper">
	<property name="serializationInclusion">
		<value type="com.fasterxml.jackson.annotation.JsonInclude.Include">NON_NULL</value>
	</property>
</bean>

Another problem raised when I tried to pass json content with attribute which were not defined in my Java classes. The most obvious option is to change the implementation to add this new parameter, but I preferred to keep it unchanged and decided to manage it globally.
That’s how I did it:

<bean
	class="org.springframework.beans.factory.config.MethodInvokingFactoryBean">
	<property name="targetObject" ref="jacksonObjectMapper" />
	<property name="targetMethod" value="configure" />
	<property name="arguments">
		<list>
			<value type="com.fasterxml.jackson.databind.DeserializationFeature">FAIL_ON_UNKNOWN_PROPERTIES</value>
			<value>false</value>
		</list>
	</property>
</bean>
<mvc:annotation-driven >
	<mvc:message-converters register-defaults="true">
		<bean class="org.springframework.http.converter.json.MappingJackson2HttpMessageConverter">
			<property name="objectMapper" ref="jacksonObjectMapper"/>
		</bean>
	</mvc:message-converters>
	
</mvc:annotation-driven>

Implementing CSRF protection with Angular-js

After 2 articles on the security, I continue with the set up of the CSRF protection using Spring security.

The main idea behind is to prevent some one else to create (forge) a request without our authorization. Imagine that a page uses JavaScript to send something like http://my.bank.com/pay/15000/cayman_account, if you are connected to your bank in another tab browser, the request could be successful…
To avoid that, for each request, the server sends a unique identifier (Spring uses a UUID) and it must be part of the request you send back to the server.

Fortunately, Angular provides a support for CSRF protection. Following the documentation, the server needs to provide a cookie named XSRF-TOKEN and Angular will send its value in a header named X-XSRF-TOKEN. On the server side, Spring then verifies that the value from the header is the expected one and continue the process, otherwise, it sends an error.

But to make them working together, we need to make some adjustments.
I use the same pet store application I used in my previous article and it is still available on github.

Let’s go to the code!
First of all, on the server side, Spring provides the identifier as a request attribute, because in a standard Spring-MVC application, it is very simple to add it in an HTML form using expression language. But with our SOA approach, we are just sending data, therefore we need to put it in the cookie we send to the browser. To do that, I defined a filter which just take the CSRF token and set a cookie:

@Component
public class CsrfInjectorFilter extends OncePerRequestFilter {

	private static final String XSRF_TOKEN = "XSRF-TOKEN";
	private static final Logger LOG = LoggerFactory.getLogger(CsrfInjectorFilter.class);

	@Override
	protected void doFilterInternal(HttpServletRequest request, HttpServletResponse response, FilterChain chain)
			throws ServletException, IOException {

		CsrfToken csrfToken = (CsrfToken) request.getAttribute(CsrfToken.class.getName());
		if (csrfToken != null) {
			Cookie cookie = new Cookie(XSRF_TOKEN, csrfToken.getToken());
			//mandatory to avoid duplicate cookies in browser
			cookie.setPath("/");
			response.addCookie(cookie);
		}else {
			LOG.debug("No csrf token found!!");
		}
		chain.doFilter(request, response);
	}
}

Note: I choose to define the cookie name as Angular expects it, but it also possible to define its name on the client side (using $httpProvider.defaults.xsrfCookieName) and keeping the web service ignore who is the client.

There is also one important thing you must absolutely do (I spent half of a day with that!), you have to set a path to your cookie! Otherwise, for some reasons, the browser does not replace the cookie when it is changed, and it simply manage several cookies with the same name, resulting in CSRF errors on the server side. I tested this issue with Firefox and Chrome.
On the other side, I need to configure Angular to return the header as expected by Spring, this is done in the app.js file:

petstoreApp.config(['$httpProvider', function($httpProvider){
    $httpProvider.defaults.xsrfHeaderName = 'X-CSRF-TOKEN';
}]);

We can see that the code is very straightforward.

Angular-js and Restful web services access

Following my previous article about the integration of Spring Security with web services, I will now explain how I integrate it with an Angular-js project.
For that purpose, I am a little bit lazy and I use a project I already written instead of building a completely new one.

The project is a “funny” pet-store based on Spring boot (I like it definitely!), using an H2 database and Flyway to restore its content at each start.
The full project is available on Github.

The web services are defined with Spring MVC in the net.classnotfound.pet.web.PetController:

package net.classnotfound.pet.web;
...
@RestController
public class PetController {
	
	@Autowired
	private PetService petService;
	
	@RequestMapping(value = "/pet/all", method = RequestMethod.GET)
	@ResponseBody
	public Collection<JsonPet> findAll() {
		...
	}
	
	@Transactional(propagation= Propagation.SUPPORTS)
	@RequestMapping(value = "/pet/{id}", method = RequestMethod.GET)
	@ResponseBody
	public JsonPet find(@PathVariable("id") final Integer id) {
		...
	}

	@Transactional(propagation= Propagation.REQUIRED)
	@RequestMapping(value = "/pet", method = RequestMethod.POST, consumes = {"application/json" })
	@ResponseStatus(HttpStatus.CREATED)
	public @ResponseBody JsonPet save(@RequestBody final JsonPet jsonPet) {
		...
	}
}

The PetService is the class responsible of business processes (mainly access data here).

We can see that there are 3 actions available to:

  • display the list of pets in the store
  • display the detail of a pet
  • create a “new” pet in the store

The attributes of the requestMapping annotation seem explicit, for example, with @RequestMapping(value = “/pet”, method = RequestMethod.POST, consumes = {“application/json” }), I express that the method can be reach with /pet URL, by using the POST method and it needs json.
Based on Spring, the conversion from Json to Java Object (and the opposite) is totally hidden (it relies on Jackson API) and it avoid this burden to the developer (meaning me, here!).

Now that the landscape is set, let’s have a look at the security configuration (it should be familiar to whose who read my previous article):

@Override
protected void configure(AuthenticationManagerBuilder auth) throws Exception {
	auth.inMemoryAuthentication().
	  withUser("admin").password("password").roles("ADMIN", "USER").and().
	  withUser("user").password("password").roles("USER");
}

@Override
protected void configure(HttpSecurity http) throws Exception { 
	
	http
	.csrf().disable()
	.authorizeRequests()
	.antMatchers("/login").anonymous()
	.antMatchers(HttpMethod.GET, "/pet/**").access("hasRole('USER')")
	.antMatchers(HttpMethod.POST, "/pet/**").access("hasRole('ADMIN')");
	
	http.exceptionHandling().authenticationEntryPoint(authenticationEntryPoint);
	FormLoginConfigurer formLogin = http.formLogin();
	formLogin.loginProcessingUrl("/login");
	formLogin.usernameParameter("username");
	formLogin.passwordParameter("password");
	formLogin.successHandler(authenticationSuccessHandler);
	formLogin.failureHandler(authenticationFailureHandler);
	
}

I started with the users definition with their roles.
And after that, Y can see how I restricted access to the features, as previously, I disabled the CSRF protection but I will come back to it in a next article.

Now, if a unauthenticated user tries to access the protected resources, he will receive an HTTP 401 error. Let’s see how I deal with it on the client side (Angular-js).
The web pages are located in

The idea is to have the generic process to intercept the right HTTP code when the user needs to be authenticated, and redirect him automatically to the login page .
The application is based on the single page principle, for that purpose, I use the ngRoute module, it is configured in the app.js file:

petstoreApp.config(['$routeProvider',
  function($routeProvider) {
    $routeProvider.
      when('/pets', {
	      templateUrl: 'partials/pet-list.html',
	      controller: 'PetListCtrl'
      }).
      when('/pets/new', {
    	  templateUrl: 'partials/pet-new.html',
    	  controller: 'PetNewCtrl'
      }).
      when('/pets/:petId', {
          templateUrl: 'partials/pet-detail.html',
          controller: 'PetDetailCtrl'
      }).
      when('/login', {
          templateUrl: 'partials/login.html',
          controller: 'LoginCtrl'
        }).
      otherwise({
        redirectTo: '/pets'
      });
  }]);

As expected, we can find our 3 pages related to the web services features and another one to ask credentials to users. This one will be used when redirecting unauthenticated users.
For that purpose, the application is configured with an Angular-js interceptor. All the requests sent or received can be manipulated by interceptor, this is very useful for common purposes like error management or authentication, and it is exactly the way I use it (still in the app.js file):

petstoreApp.factory('errorInterceptor', ['$q', '$location', '$window', '$injector', function($q, $location, $window, $injector) {
	var authSrv;
	var popupSrv;
	var errorInterceptor = {
			'responseError' : function(rejection) {
				var status = rejection.status;
				if (rejection.data){
					if (status == 401) {//Unauthenticated, redirect to login page
						$window.location.href="/#/login";
						return;
					} else if (status == 418) {//undefined error->HTTP code 418 ( I'm a teapot) is responsibility of the caller
						return $q.reject(rejection);
					} else{
						popupSrv = popupSrv || $injector.get('popUpSrv');
						popupSrv.alert("Erreur: "+rejection.data.message+ (rejection.data.stackTrace?"\r\n___________________________________\r\n\r\nErreur complète:\r\n" + rejection.data.stackTrace:''));
						return;
					} 
				} else {
					popupSrv = popupSrv || $injector.get('popUpSrv');
					popupSrv.alert("Erreur inconnue( http status: "+status+")");
					return;
				}
				return $q.reject(rejection);
			}};
	return errorInterceptor;
}]);
petstoreApp.config(['$httpProvider', function($httpProvider) {  
    $httpProvider.interceptors.push('errorInterceptor');
}]);

The interceptor is in charge of managing the HTTP 401 code by redirecting to the login form, and also of the display of a generic message error.
For more information about interceptors and their usage, you can directly have a look at the official site.

And now, it works! The web site can access resources protected with Spring security. In the real world, I should display the new pet button only for admin users, it means that I should also provide a service indicate which actions are available for a user.

For those who missed the point, the sources of the project are available here.

Securing Restful web services using Spring-Security

After several months of silent, I woke up… With some security interests!
The security is a common requirement on many web projects, and fortunately, Spring helps us to implement it.
The most common way to do it is just defining a set of rules and letting Spring manage the login and so on.
In this demo, I will use Spring-boot (for its simplicity) but the security configuration is exactly the same with a web application.
And to be clear, I do not have the pretentiousness that this is THE way to follow when you need to secure your web services and I am convinced that someone else found another way to do it, but this is my solution, I used it several times, and it works perfectly, so maybe it will help someone else… One day 🙂

To start simply, I just built a simple web site with 2 pages:

  • the home page (/index.html), which is freely accessible
  • and another page (/secured/page.html) which is in a restricted area

Using Spring-boot, the structure is something like:

structure

By default, Spring boot published everything under the static directory and is accessible via http://localhost:8080/

For instance, everybody can access to our pages, it’s time to remedy to that state.

Let’s start with the interesting part of the Maven Configuration:

<parent>
	<groupId>org.springframework.boot</groupId>
	<artifactId>spring-boot-starter-parent</artifactId>
	<version>1.3.2.RELEASE</version>
	<relativePath/> <!-- lookup parent from repository -->
</parent>

<properties>
	<project.build.sourceEncoding>UTF-8</project.build.sourceEncoding>
	<java.version>1.8</java.version>
</properties>

<dependencies>
	<dependency>
		<groupId>org.springframework.boot</groupId>
		<artifactId>spring-boot-starter-security</artifactId>
	</dependency>
	<dependency>
		<groupId>org.springframework.boot</groupId>
		<artifactId>spring-boot-starter-web</artifactId>
	</dependency>
	
	<dependency>
		<groupId>org.springframework.boot</groupId>
		<artifactId>spring-boot-starter-test</artifactId>
		<scope>test</scope>
	</dependency>
</dependencies>

<build>
	<plugins>
		<plugin>
			<groupId>org.springframework.boot</groupId>
			<artifactId>spring-boot-maven-plugin</artifactId>
		</plugin>
	</plugins>
</build>

The starter dependencies are somehow facilitators provided by Spring boot to add some dependencies, I think that we can guess their purpose just by their name. For more information about this pom, you can have a look to the Spring-boot site.

Now, we have to restrict access to the secured part of the site, this is done here:

@Configuration
@EnableWebSecurity
public class SecurityJavaConfig extends WebSecurityConfigurerAdapter {
    @Override
    protected void configure(AuthenticationManagerBuilder auth) throws Exception {
        auth.inMemoryAuthentication().
          withUser("admin").password("password").roles("ADMIN", "USER").and().
          withUser("user").password("password").roles("USER");
    }
 
    @Override
    protected void configure(HttpSecurity http) throws Exception { 
        http
        .authorizeRequests()
        .antMatchers(HttpMethod.GET, "/secured/**").access("hasRole('USER')");
        
        //indicate that we are user login form authentication
        http.formLogin();
    }
}

For the demo, I define my users directly in the application, this is AuthenticationManagerBuilder.inMemoryAuthentication().

Inside the configure(HttpSecurity http) method, you can see that I just defined the URL of the secured part, indicating who can access it based on role.
And finally, I indicate that I want to use the form login authentication as it is provided by Spring.

According to the configuration, the users can access freely the home page, but only the users authenticated and with the right role can access to the secured part. This is a perfect configuration for a standard web site, but remember that we nee to secure a web service, the redirection to a login form is not an acceptable way for that purpose and we need to customise this behavior.

The first issue to solve is to avoid to redirection to the login form in case of unauthenticated access, I do it by changing the AuthenticationEntryPoint with mine:

@Component
public class RestAuthenticationEntryPoint implements AuthenticationEntryPoint {
	@Override
	public void commence(HttpServletRequest request, HttpServletResponse response, AuthenticationException authException)
			throws IOException, ServletException {
		response.sendError(HttpServletResponse.SC_UNAUTHORIZED);
	}
}

As you can see, this is very simple, I just need to send the right HTTP code to inform my client that it needs to authenticate before accessing the resource it requested.
And to indicate to Spring that I slightly changed the authentication, it is very complicated because… Well, it’s Spring, so, I just need to add the following in my SecurityJavaConfig:

[...]
@Autowired
private AuthenticationEntryPoint authenticationEntryPoint;
[...]
protected void configure(HttpSecurity http) throws Exception {
	http.exceptionHandling().authenticationEntryPoint(authenticationEntryPoint);
}

Now, when someone… As we are dealing with web services,I should say something tries to access to secured resources, it just receives an unauthorized HTTP code instead of a redirection to the login page.
That’s a good point but we have also to manage the successful and erroneous authentication. Instead of redirecting to a page, we have to inform the client that the credentials are valid or not .
For each case, we need to define two different classes:

  • in case of success, we let Spring clean-up the temporary data:
    @Component
    public class RestAuthenticationSuccessHandler extends SimpleUrlAuthenticationSuccessHandler {
    	@Override
    	public void onAuthenticationSuccess(HttpServletRequest request, HttpServletResponse response,
    			Authentication authentication) throws IOException, ServletException {
    		clearAuthenticationAttributes(request);
    	}
    }
    
  • in case of error, we return an HTTP code and an optional message:
    @Component
    public class RestAuthenticationFailureHandler implements AuthenticationFailureHandler{
    	@Override
    	public void onAuthenticationFailure(HttpServletRequest arg0, HttpServletResponse arg1, AuthenticationException arg2)
    			throws IOException, ServletException {
    		arg1.sendError(HttpServletResponse.SC_FORBIDDEN, "Wrong username or password");
    	}
    }
    
  • And then the last step is to add them to the Spring configuration:

    [...]
    @Autowired
    private AuthenticationFailureHandler authenticationFailureHandler;
    
    @Autowired
    private AuthenticationSuccessHandler authenticationSuccessHandler;
    [...]
    protected void configure(HttpSecurity http) throws Exception {
    	[...]
    	FormLoginConfigurer formLogin = http.formLogin();
    	[...]
    	formLogin.successHandler(authenticationSuccessHandler);
    	formLogin.failureHandler(authenticationFailureHandler);
    
    }
    

    Now, we can define our web services, and Spring will manage authentication for us.
    To protect your services, obviously, you need to add something like:

    http
    .authorizeRequests()
    [...]
    .antMatchers(HttpMethod.GET, "/secured/services/**")
    .access("hasRole('SERVICE_USER')");
    

    And you can also restrict access by the HTTP methods, someone can read data (GET method) but could not be allowed to modify it (POST/PUT methods).
    That’s it!

    In a next post, I will show how I used it with Angular-js, stay tuned 😀

    (the full project is available on Github)

Add CORS management with Tomcat and Angular-Js

In one of my projects, I decided to setup a SOA architecture using Angular-Js as the UI layer. My main focus is to have a clear separation between the business and the presentation. Another benefit of this is that it is possible the use different servers to these 2 parts.
But doing this causes an issue, because of security, an Ajax request cannot connect to a server which is in another domain from where it is originated. It is done like that to avoid some malicious code executed in a page sends information to another server but in my case, it’s a problem. This behavior is called the Cross-origin Resource Sharing and fortunately, most modern browsers (it excludes IE) can deal with this but the server needs more configuration.
On the server side (Tomcat), there is a provided filter that we need to add to our web configuration (CorsFilter).
In the web.xml, we add:

<filter>
    <filter-name>CorsFilter</filter-name>
    <filter-class>org.apache.catalina.filters.CorsFilter</filter-class>
    <init-param>
        <param-name>cors.allowed.origins</param-name>
        <param-value>*</param-value>
    </init-param>
    <init-param>
        <param-name>cors.allowed.methods</param-name>
        <param-value>GET,POST,HEAD,OPTIONS,PUT,DELETE</param-value>
    </init-param>
</filter>

I added also some parameters but the default values should be enough.

Now, the server can manage the headers sent by the browser and with this configuration, the filter will automatically allow the request, it is also possible to configure the filter to restrict access to certain domains, but in my case, as I am already in a restricted network, it is not needed.

Another issue I encountered was with the web session. My services need an authenticated user, it is done using an authentication page and a web session. But when the CORS principle is used, by default, the browser does not send the session cookie, to enable it, I added in my Angular app:

var classNotFound = angular.module('classNotFound', ['ngRoute', 'ui.bootstrap', ...])
	.config(['$httpProvider', function($httpProvider) {//mandatory to enable cors feature (http://en.wikipedia.org/wiki/Cross-origin_resource_sharing)
		$httpProvider.defaults.withCredentials = true;
	}])

Now it works!

Add global exception management in Rest web service

Since I adopted a Restful architecture, I need also a simple way for the exceptions management.
With Spring, it can be easily set-up with in a central class, avoiding a painful management per service.

To do this, we need to define a class with the @ControllerAdvice annotation, this class will be automatically associated to our controllers during the classpath scanning. By default, it will be added to all controllers found, I decided to restrict to our concerned classes by specifying the controller’s package.

package net.classnotfound.exception.handler;

import[...]

@ControllerAdvice(basePackages = {"net.classnotfound.controller" })
class ControllerExceptionHandler {

    private static final Logger LOG = LoggerFactory.getLogger(ControllerExceptionHandler.class);

    @ResponseStatus(HttpStatus.INTERNAL_SERVER_ERROR)
    @ExceptionHandler(RuntimeException.class)
    @ResponseBody
    String handleNullRequest(final HttpServletRequest req, final RuntimeException ex) {
        LOG.error(ex.getLocalizedMessage(), ex);
        return ex.getLocalizedMessage();
    }

    @ResponseStatus(HttpStatus.BAD_REQUEST)
    @ExceptionHandler(Exception.class)
    @ResponseBody
    String handleBadRequest(final HttpServletRequest req, final Exception ex) {
        LOG.error(ex.getLocalizedMessage(), ex);
        return ex.getLocalizedMessage();
    }

}

The @ExceptionHandler defines the method as a specific handler for the type of exception, we have to use the same type of exception in the method signature.
The @ResponseStatus indicates which HTTP status is returned for this particular type of exception.