User Migration to WSO2 Identity Server is Simple and Easy now!!

4 min readAug 25, 2019

Users are the most significant element in a system. If you are running a system right now, you might have stored users in a user store. This includes their username and password along with some basic information of a user such as email, mobile, first name, last name, address, NIC/SSN, date of birth, user_id and etc in a single entry.

Why you need a central Identity Server system like WSO2?

This is because in old systems if you have 3 different web sites/applications, you have to manage 3 different user bases, to login each of the applications. This can cause lot of user duplication and waste of resources.

Hence we use a centralize Identity Server System which manage all the users at one place(user-store)

How to migrate users to WSO2 Identity Server User stores?

WSO2 identity Server supports different user stores. Out of them LDAP, AD and JDBC are the famous and commonly used by our clients.

If you already have a LDAP or AD userstore you can directly plug your LDAP or AD to the WSO2 Identity Server.
But if you are using JDBC userstore, you need to migrate to our JDBC userstore schema. Because our JDBC user store schemas can be different. You can check the userstore schemas from files in <is-home>/dbscripts/ folder. To migrate you need to follow any of the below migration strategy.
If you wish to migrate from a LDAP userstore to JDBC userstore, you also need to follow any of the migration strategy below.

Migration strategies

Method 1: Bulk User Import

In WSO2 Identity Server we have a UI option to upload set of users at once in a csv file rather than creating each manually. If the option to import users in bulk is not enabled in your product by default, you can enable it by adding the following property to the JDBC user store configured in the user-mgt.xml file (stored in the <PRODUCT_HOME>/repository/conf directory). Please see the User Store management section for more information.

<Property name="IsBulkImportSupported">true</Property>

It is recommended to upload a maximum of 50,000 users at a time. If you need to upload more users, you can upload them in separate batches of 500,000 each. You can also specify the size of the file that you can upload to the product in the <PRODUCT_HOME>/repository/conf/carbon.xml file using the TotalFileSizeLimit element as shown below. This value is in MB.

<TotalFileSizeLimit>100</TotalFileSizeLimit>

You must first create a CSV file or an Excel file with the user information. It is possible to import the username and password directly from the CSV/Excel to the product. Other user attributes can be imported if claim URls are defined for such attributes. Shown below are the claim URls that are defined be default in WSO2 IS. These will allow you to import the user’s email address, country, given name etc. in addition to the username and password.

The username, password and other attributes (claim URls) that you import should be given in a CSV file as shown below. Note that the first line of the file will not be imported considering that it is not a username.

UserName,Password,Claims

name1,Password1,http://wso2.org/claims/emailaddress=name1@gmail.com,http://wso2.org/claims/country=France

name2,Password2,http://wso2.org/claims/emailaddress=name2@gmail.com,http://wso2.org/claims/country=France

name3,Password3,http://wso2.org/claims/emailaddress=name3@gmail.com,http://wso2.org/claims/country=France

If you do not want to give a password by default, you can enable the Ask Password option for the server, before user import.

Method 2: User Migration using java client

If the number of users exceeded more than 20,000 there can be issues when uploading users via the UI of the WSO2 Management console.

In order to avoid the UI timeout issue, we have introduced a new serial java client which does the same task of bulk user import.

You can find the java client publicly available in https://github.com/wso2/samples-is/tree/master/bulk-user-import-sample

By following the instructions in ReadMe you can try out the java client easily.

NOTE: If an error occurs in the middle of the import, the execution will stop and the line of the .csv file where error occurred will be printed in the terminal. Which means that the user contained in that line was not imported, neither any of the next users in the file. Users processed until the error has occurred must have been imported normally. Example:

User where error occurred: 
name00001,Password1,http://wso2.org/claims/emailaddress=name1@gmail.com,http://wso2.org/claims/country=France

Method 3: Run-Time user migration

If it is really challenging to migrate users via a .CSV file we can try this approach. If you have number of existing user stores and if you have duplicate users in each user store, then this approach is the ideal approach to migrate users to the WSO2 User store without duplicating users.

In this approach we need to write a custom user store as in [1]. The logic will work as follows,

First all the available user stores should be plugged as secondary user stores.
When user will be authenticated, they will be first authenticated from WSO2 Carbon user store.
If user is not available in WSO2 Carbon DB, then user will be authenticated from all the other user stores, consecutively.
If user got authenticated from any other user store, the user will be added to the WSO2 Carbon Primary User store.

When we follow this procedure, after about 1 to 2 years, all the users will be migrated to the WSO2 Carbon Primary DB. The time taken will depends on how frequency the users will be interacting to the WSO2 Identity Server.

[1]https://docs.wso2.com/display/IS580/Writing+a+Custom+User+Store+Manager

Hope you clearly got an idea how to migrate users from a different Identity Server to WSO2 Identity Server. WSO2 IAM is really flexible to use as your requirement. Hence try this and let us know if have more questions regarding these strategies.

Thanks :)