When a business wants to provide a service via their website or web application, they want to ensure that their users experience a good service without compromising on security. By building in a focus on security from the start when building a website or application, it is possible to ensure a quality and secure user experience.

This focus on security can be started with a focus on a set of values – the pillars of information security as proposed by OWASP;

1. Maintain confidentiality by only permitting user access to appropriate data
2. Maintain integrity by ensuring that users are only allowed to view or change appropriate data
3. Maintain access by ensuring that functions are only available as and when users require them

Continue reading…

When I was completing my scrum master training with Jean Tabaka, we started the course with an interesting discussion on what we thought Agile development and Scrum was and wasn’t about. This discussion illustrated some valuable points for project teams that in today’s business world, have to be more responsive, flexible and creative when delivering software solutions.

Continue reading…

When you browse a website, intuitively or otherwise, you are judging the usability of the site. To paraphrase the ISO definition – usability is a measure of the satisfaction, efficiency and effectiveness with which a user completes specified goals when using the website. When viewing easy to use websites, there is a common trait, in that they were designed with their user audience in mind.

Continue reading…

I would regard a quality website as a site that provides an excellent user experience for it’s visitors. In this blog, I write about some important steps that are required to define the requirements for such a website.

Step 1 – Agree a vision for the website

A vision for a website is important because it defines the scope and intent of the site. A good vision statement has the following characteristics;

• It includes a value and a measurable goal
• It is a concise statement
• It should be closely related, if not the same as the business vision statement
• It is written in the present tense and helps the visitor to visualise

Continue reading…

I was one of a number of members of the Irish Computer Society’s security professionals network that submitted a submission recently to the Department of Justice following their recent publication of a consultation paper on data protection.

The general consensus of the response was that the option involving the development of a code of practice incorporating best practices on systems development and security and promoting greater awareness under existing legislation was the preferred option.

This code of practice could be the basis of a voluntary standard promoted by the Data Protection Commissioner to encourage adoption of best practices by publicly recognising organisations that adopt the standard and demonstrate compliance.

In a previous post I wrote about a the Plan – Do – Check – Act approach to developing an appropriate social media strategy for your business. I want to follow-up on that post with a list of case studies showing practical examples of companies around the world of how they are using social media applications and services to enhance their service.

In the previous post, I wrote that social media applications can be used internally within an organisation to enhance knowledge management and collaboration as well as external uses to directly enhance customer service. Both have a similar end goal – service to the customer should improve through direct access to information and/or through a better informed staff. Continue reading…

How can a business decide on using the cloud or not ?

Cloud computing services are increasingly being used worldwide. However there are still businesses, of all sizes, looking at cloud computing and wondering if it is appropriate for them and what benefits and costs will it involve. In fact, they are wondering how real is the cloud for them. Looking at the variety of comment on the use of cloud based services, you can see the common questions or concerns;

• How secure is my business data ?
• What level of service will my business receive ?
• Are there any hidden costs ?
• How easy can I scale my computing needs to support my business ?
• Am I liable to break any regulations such as data protection ?

As the concept and range of services being provided via the Cloud is still evolving, this wariness is understandable, but help is at hand.

The Cloud Security Alliance, a non profit organisation that was formed to “promote the use of best practices for providing security assurance within Cloud Computing”. The Alliance have produced a guide which I believe will be useful for businesses looking to assess the impact of cloud computing and avail of any opportunities.

I would like to summarise some of the key steps and recommendations that are in the the document in relation to the typical concerns and the full report is available here. Before going into some of the details, I would say that two of the strengths of the CSA guide is the level of detail provided on cloud computing and the grouping of recommendations to deal with each of the typical concerns. Their recommendations can be used as a process to help businesses manage the risks and reap the benefits.
Continue reading…

Transformation is regarded as the major part of the ETL process, where all the extracted data is cleansed and transformed into the appropriate detail for the final loading into the data warehouse. Starting with cleansing the data, when loading data from a number of different transaction systems, it is important to go through the following steps to ensure that high quality data is being prepared for the data warehouse by;

• Standardise descriptions, attributes and currencies. For example, in one transaction system, ‘Customer Service’ could be referenced while in another system, it may be known as ‘Customer Relations’. Similarly if data from different countries is being amalgamated, ensure that a common currency and an agreed exchange rate is used. If a garden rake is considered part of the garden equipment group, then this group attribute should be used consistently in all references to the garden rake, irrespective of the system that the particular data is taken from.
• Correcting data and spelling. For example, ensure the correctness of data by using one standard, all references to RB Consulting should be loaded as ‘RB Consulting’ and so any references to RBConsulting or RBC should be converted. Spelling is important, as transaction data is taken from systems where there can be a high volume of manual data entry and errors can occur!
• Complete a de-duplication exercise by either selecting whatever record is deemed the best quality or merging similar records. The de-duplication can require judgment and input from staff familiar with the data, particularly if for example relevant data such as address or date of birth is missing from the transaction record. However database scripts can be used to isolate and group this suspect data for analysis.
• Consistent data types are important – sometimes when data is extracted, it may not be in a consistent data format. For example, dates of birth as characters rather than date format, order amounts as characters, rather than as numerical format. These inconsistencies can be resolved either in the Extract or Transformation stages.

Calculating and deriving data

Not all of the data that you require for your analysis will have been extracted directly from the transaction systems; additional data may be required by aggregating and/or completing calculations using the extracted data. For example, using data on total sales and total customer returns, you may calculate a ‘Sales net of returns’ figure.

The amount of aggregation that will happen during the Transformation process is be driven by the level of detail required for analysis in the data warehouse (i.e. the ‘fact grain’). If high level queries with a minimal need for drilling down into the individual details, then data can be aggregated during the transformation process.

Dimensional modelling – the final transformation

As was noted earlier, data in online transaction processing systems is designed in a normalised manner so as to structure the data tables for efficient transaction processing. A dimensional data model, consisting of facts and dimensions is required to enable efficient data analysis and querying in a data warehouse where;

• Facts consist of the data that the business is trying to measure, such as sales, salaries, stock levels or profit
• Dimensions are the attributes with which a business is analysing the data, such as time, geographical regions, or products

Another perspective of the difference between Facts and Dimensions is that Facts are usually numeric and Dimensions are hierarchical (e.g. Time: Year, Quarter, Month)

A star schema is one approach to creating your dimensional data model, where a fact table is in the centre of your star and each of the required dimension tables are at the points of the star.

With the transformation of the data into a dimensional model in the staging area, it is now ready to be Loaded into the data warehouse – the final step of the ETL process.

In this post I would like to describe some of the important tasks and considerations when developing a reliable extract – transform & load process as part of your data warehousing project. In a previous post, I described Data warehousing (DWH) “as the means with which data is delivered from different operational systems such as Purchasing, Stock or Sales systems for the business intelligence process”. In this post, we will focus on the E – extract step with details on the T & L steps to follow in the next posts.

Data warehousing involves bringing information together from disparate transaction systems, each of which are organised for online processing of small amounts of transactions by a large number of individual users. The ETL process enables this transaction data to be modified in terms of its structure and detail for analysis where large volumes of data are queried by a relatively small number of users.

Extracting the data

Before extracting any information, it is important to ensure that the correct data is being sourced in the appropriate ‘master’ system. For example, customer id and name may be available in three different systems, so let’s ensure that we draw the customer information from the system where it is most complete and up to date. By compiling a map of the source systems, the map helps to ensure that the right data is drawn from the correct ‘master’ system by containing; information on;

• Each of the ‘master’ systems
• The data that is provided by each system
• The format of the data
• Frequency of change
• The relevant database tables

When data is being extracted from a number of different transaction systems as part of the ETL process, this data is stored in a ‘Staging’ database.

Keep the data consistent

Consistency is another key requirement – when extracting data from different systems, it is important that that the data is valid and accurate. For example, if sales data is being extracted from a sales transaction system where the end of day update routines are still being run, taking in sales figures from different sales locations, then that sales data extract may not be accurate or fully up to date.

It is important to ensure that the data extraction happens after all transaction updates have been completed and that each transaction is date/time stamped to help with any date/time analysis that may be occurring later in the data warehouse.

There are a number of quality checks that can be applied to the extraction process;

• Using count functions to ensure that the same amount of extracted data exists in the Staging database as the relevant transaction systems
• The use of date/time data such as last business day or last transaction date can be used as a comparison with individual transaction date/time stamps to ensure that the extraction has been completed and no data from a different business period/day has been taken in error

A selection of tools to help you

While it is possible to develop a series of extraction scripts from scratch in Java or Php, there are a range of available tools for completing data extraction and most of these tools provide both a visual gui interface as well as a scripting capability for designing and completing the extraction functions, including;

• Microsoft’s sql server integration services 2008
• Pentaho’s data integration services
• Oracle’s data integrator
• Sas data integration
• IBM Cognos 8 data manager

Once all the necessary data has been extracted into a staging database, data transformation can begin. Transformation is regarded as the major part of the ETL process, where all the extracted data is cleansed and transformed into the appropriate detail for the final loading into the data warehouse.

In a previous post, I discussed how disaster recovery planning can give a business a competitive edge by helping to identify risks and how best to ‘manage’ them. Two of the potential approaches; risk transference and risk mitigation can involve using the services of a Disaster Recovery (DR) service provider. In this post I write about some of the important requirements that you will need to consider when using such services.

Different types of DR services

There are a wide variety of DR services providers, in terms of their size, degree of specialisation and scope of services. This is in line with the varying needs of companies; some companies require a secure facility to store a backup tape and access to a single server to restore the backup in case of emergency within twenty four hours, alternatively other companies may require more immediate access to data and a range of servers to host a number of different applications.

In the previous DR related post, I spoke about completing a risk assessment, including a business impact analysis which will provide you with a list of risks and a ranking for each risk according to the company’s potential exposure (business impact x probability of occurrence). Based on the threshold decided by the company, the management of those risks with the highest exposure can then be part of the offering from the DR services provider.

Continue reading…