How much time do you usually spend securing a new system? Does security play a major role in your software architecture? How much security is enough?

Securing web applications traditionally consists of two components referred to as authentication and authorization. Authentication is wrapped up in the log-in process, during which your system needs to identify the user or other system attempting to gain access. Application developers will create log-in pages or leverage Single Sign-On solutions to address the authentication issue. In addition, HTTPS will be used to encrypt the communication between your end-users and the system. Authorization, on the other hand, is the process by which the system verifies that a user (or external system) has the permission to execute a particular function within the system. This may be addressed by implementing a role-based security model.

Is this enough? Well… that all depends on what kind of system you need and what kind of legal implications apply (such as HIPAA).

Security must be addressed in each of the following phases of the software development lifecycle.

Requirements
During requirements gathering and scope definition, one must understand what the security requirements are. Don’t just ask for them, but lead with questions and what-if scenarios.

Architecture and Design
When defining your architecture, consider the different components that will be needed to provide adequate security. Will security be encapsulated as part of each module, or will the modules in the middle tiers assume that a different tier has taken care of security?

Development
Have you identified secure coding practices as part of your coding standards such as minimizing the attack surface and establishing security defaults?

Testing
Have you planned for security testing that addresses common attacks and checks for vulnerabilities?

Deployment
Is your connection from application to database running with more privileges than necessary? Is the right security infrastructure in place, including firewalls and intrusion detection systems?

As you’re evaluating your current system or building the next generation of products, please take a moment to think about security beyond authentication and authorization. I’m not suggesting that every system needs to be secured like Fort Knox, but rather that you consciously think about the appropriate level of security for your system and document it as part of your system architecture and development process.

There are many references out there to help you improve software security. I would recommend the Open Web Application Security Project (OWASP) to start.

Its amazing how a little discipline can go a long way on a project.  Having the mettle to ensure that regardless of how tired you are you still take the time to write your code the proper way.  Running the code one more time and stepping through it just to be sure it really works and prove it to yourself.  Each of us has that little devil voice inside our head that tells us there is no reason to compile.  Its only a small code change that was just made.  We all hear it.  Its just a matter of who's got enough experience and battle scars to know not to listen vs. those that don't.

The questions you should ask yourself before each code release go something like:

• Is someone else going to be really testing my code for the first time?
• Is my code really worthy of being on the test server?
• Would I bet my salary on what I'm doing?

Matt Goebel sat down with Larry Clarkin recently to do a podcat for The Thirsty Developer, which is a podcast that Larry and Dave Bost have started. 

The topic of the discussion was globalization and localization in Windows Forms applications.  Matt has some very relevant experience from a Smart Client that he's built that has over 15,000 seat deployments and runs in 6 different languages including Mandarine Chinese.  He and Larry swap some stories about trials and tribulations and some best practices on building applications that need to support multiple languages.

Here are some relevant links to articles by Crowe folks on Windows Forms or Smart Client related topics:
Windows Forms Tricks You May Have Missed
ClickOnce Extends Windows Forms Applications into Smart Client Applications
Build a Localized Form that Speaks the User's Language
Creating Custom WPF Controls
Getting Started with Microsoft Synchronization Services for ADO.NET
Making a Pre-Vista Application Vista Compliant

I sat down with Larry Clarkin recently to do a podcat for The Thirsty Developer, which is a podcast that Larry and Dave Bost have started.  Larry and Dave are both Developer Evangelists for Microsoft that I'm fortunate to interact with in the community.

The topic of the discussion was in regards to Visual Studio Team Suite 2008 and some of the great features in the Developer Edition and how they're helping our Crowe project teams in their effort.  As a bonus I got to say cyclomatic complexity, which while a tongue twister is fun to discuss.

Take a listen.  I'd love to hear some feedback.

When a new solution is being discussed, it’s easy to focus on all of the things it should do, what it should look like, and what technologies it should be based on. It’s very easy to get caught up in discussions with business resources about comprehensive feature sets and ways to present the information to customers. With a technical audience, it’s entertaining to squabble about the pros and cons of using one technology versus another to address the functional needs of a business problem.

When do discussions about disaster recovery, performance and scalability come into play? When should they? Why can’t I just say that the system shouldn’t fail, it should be fast, and it should be able to scale? The short answer is that this could be quite cost-prohibitive. The longer answers are below.

Let’s start with disaster recovery. I liken the conversation about disaster recovery to discussions about life insurance. To most people, it’s not fun to talk about what would happen if you die. Same goes for the brand new system you’re about to build. Disaster recovery is not a topic to be discussed AFTER the database gets corrupted or the application server CPU overheats. These are conversations that have to be had BEFORE the system is designed. Just as the functional requirements are documents, so should be these disaster recovery requirements. Based on different up-time needs and recovery efficiency, disaster recovery requirements could vary from making sure each server is backed up nightly to putting structure around a hot-backup on the other side of the country.

When it comes to talking about performance, the conversation tends to be easier; however, the proper documentation of performance requirements can be tough. How do you account for variations in network traffic? What if another process is running on your application server? The recommendation I have here is that you start by determining what an adequate response time looks like for the key functions in your new system. This needs to be balanced with the cost for delivering that kind of performance. Execute your performance tests in an environment that closely mirrors production. When it comes to documenting actual performance requirements, these can vary from a generic average response time per function to specific metrics around the key transactions of the system based on expected usage patterns throughout the day or during a specific time period (think concert ticket sales).

What about scalability? How do we expect the system to grow over the next several years? How many users do we expect initially? How many of them will be using the system concurrently? Even after having the answers to these questions (or best guesses), documenting requirements around scalability is difficult. I would recommend turning these into technical requirements around sustainable transaction throughput. Considering these requirements during the design phase allows you to build in the proper flexibility so the system can support the future anticipated growth.

Beyond these three different types of non-functional requirements, there may be many more that should be considered based on the system’s particular needs. The trick is to recognize early on in the initiation phase all the different non-functional aspects that are critical to the success of the system, documenting them, validating them, and ensuring they are written in a objective way.

I was recently engaged in discussions about the use of design patterns that resulted in my reflecting on projects I've seen where I judge design patterns to have been used successfully.  Design patterns can indeed add significant value to a solution and its long term viability and success.  They can also add a great deal of complexity based upon the pattern in question.  The more main stream the more likely the code will pass the maintenance test where another developer can pick up the code and successfully make modifications or additions that properly adhere to the pattern in play.  The more obscure the pattern the less likely another developer can easily maintain. 

As I reflected on times where I've seen patterns be successful I kept coming back to a commonality of what I refer to as a guidance solution or guidance framework being in place early on.  This boils down to work being done up front to put in place a detailed example of how code will be written.  It involves items such as creating a solution, putting stubs for various code projects that will hold the UI, data layer, business logic, etc, Enterprise Library or other core components included, and then building out some portion of the solution that hits the database or performs some other item of functionality that will be fairly common within the code base.  An ASP.NET web project example would be getting the solution setup, establishing multiple projects to hold different tiers of code, getting the authorization provider up and going, and building a secured maintenance page that retrieves and allows updates of data from a data source.  Plugging in the use of data retrieval, error handling, audit trail, and logging, and demonstrated use of design patterns such as MVC, MVP, factory, etc to provide a concrete example.  This stubbed out solution then serves as a working example for other developers to emulate in writing their code so they aren't starting from scratch themselves.

Guidance Solutions / Frameworks certainly don't guarantee success by any means, but the times I've seen them in play from the start other developers have been able to roll on to the project and contribute fairly rapidly without an extreme learning curve.  The key being that it needs to be done up front to maximize the value and serve as an example for consistency from the start.  Otherwise there is an inevitable part of the project where refactoring is forced to take place to bring consistency to a code base riddled with inconsistency as many of the developers chose different approaches to common items such as database access, logging, and exception handling.  Having the guidance framework in place from the start helps avoid this what can be time consuming and costly quality issue phase on projects.  Even better is if the up front work is done with more common practices that are well known and will require less education within the team and a higher chance it will be willingly adopted over having to force adoption.

Its continually amazing to me how discipline can go a long way to helping ensure a solution has long term maintainability.  Making sure you set parameters and agreed upon standards at the start of a project and then having the mettle to ensure that regardless of how tirelessly you're having to write code you take the time to stay within the parameters.  More often than not when I'm reviewing a solution I see a lack of standards or consistency and excuses such as "there isn't time for it" is almost always the explanation offered or implied.  Or even more shockingly you find an experienced developer that is coding with discipline, but not caring that others aren't and not sharing their best practices.  Open forum peer reviews and discussion of code is something I am a big proponent of in projects especially early on.  It helps ensure that folks are on the same page in terms of style, practices, and that the solution is getting off to the right start.  This is great for reviewing key and common elements of a solution even if it is only practiced early in the project and then abandoned as the project is more mature in the lifecycle.  The downside though is that it isn't realistic to expect that all code can be reviewed by such a process as there is often way too much code.

Enter tools.  Even if there are no explicit standards being followed, there are code analysis and metric tools that can be a big help to making sure the overall solution is more maintainable and performant.  Code analysis will examine the code and look for situations where items may be less than optimal or could be done another way for better performance, maintainability, security, etc.  Common situations are things such as over use of string concatenation where specially designed and class library provided string building objects would be better or inconsistent use of naming conventions.

Code metric tools will examine things such as the overall number of lines of code, depth of inheritence, and cyclomatic complexity which is the number of branches that can be taken within a specific block of code.  The higher the cyclomatic complexity the harder it will be to unit test the code as there are more conditions to test.

There are a number of tools out from vendors such as IBM (Rational), Compuware (DevPartner), and Microsoft (VSTS).  Microsoft recently released the Developer edition of its Visual Studio Team Suite that now includes code metrics and analysis.  I've been experimenting with it and using it to look at a bunch of different code and I've been pretty happy with it and its ease of use and understanding so far.

I highly recommend that developers and development teams adopt a tool to help automate code reviews and perform code analysis and metrics if you're not already.  Even the most disciplined developer will benefit from having an automated review of their code.  It only takes a couple of minutes based on the tool and size of the code base and the results can give you plenty to think about and decide to take action on or not.  Its a great way to learn additional best practices you may not have been aware of by examining the output of the automated review.  Each developer should be responsible for running such as tool against their code and remediating issues prior to having flagged an item as complete.  Again, it is a decision making process as you won't want to blindly make all changes suggested, but such a process prior to turning in code will go a long way to helping overall maintainability.

If your IT organization is one of the many experimenting with a simple SharePoint implementation, you'll agree that the out-of-the-box functionality is quite amazing. You can enable your business users to quickly manage their own content, setup workflows, and collaborate. The part that gets interesting is when you're ready to actually start building applications on top of SharePoint. For example, the publication and approval process for content changes doesn't require you to maintain multiple environments, such as development, test, staging or integration environments. Your users are making changes in production! That kind of behavior is severely frowned upon in traditional application development. How do you introduce simple applications in SharePoint while at the same time ensure your business users continue to enjoy the same level of freedom for managing their content?

It all starts with understanding the difference between what Microsoft calls “Artifact Development” and “Assembly Development”. For those of you who are familiar with Software Development Lifecycle processes, the transition to “Assembly Development” will feel very comfortable. It consists of developing code, packaging it up, and deploying it through various test environments before deploying in production. The “Artifact Development” process will require some behavioral changes on behalf of your users that had the freedom over your content. Instead of making content changes directly in production, you’re now asking them to follow a more rigorous process where they’re packaging up their content changes using SharePoint Designer. To understand more about these how to join these two processes, check out the following article on MSDN: Team-Based Development in Microsoft Office SharePoint Server 2007

Before investing too much time and energy in building out formal processes and environments, determine how much SharePoint development your organization will really be doing. Are you just considering building a few features that extend some common behavior, or is SharePoint going to be your platform for a custom workflow solution? How robust is your current application development methodology, and how can you best leverage the tools and processes you already have?

Once you’ve thought about these questions, you can start building the plan for your own customized SharePoint development process.

The Solutions Architecture blog is now officially back on track!  This will be my forum to hop up on a soap box about various technology related solutions and the involved architecture.  Architecture is one of those terms that is often used but has a different definition depending upon who you're asking and in what context.  But that is precisely why it is the focus of this blog.  The content will be based upon interactions with Crowe, clients, and partners such as Microsoft as we look to solve business problems with technology.  Some discussions are things that will be ultimately powerful due to simplicity and others elegant and unique.

Some prior content has been migrated, but overall it's a fresh start!

As we were designing out the myColts.net site we went through a lot of possible design concepts. We were chasing after the ability to be able to support a site where you could radically changes the look or layout without a ton of effort. The Colts are in the entertainment business, so there is an extreme need to be fast and nimble and change on a dime to take advantage of opportunity. Do you use content management and templates? Use concepts similar to DotNetNuke where there is one page that loads webparts dynamically? During design exploration I just kept coming back to thinking about SharePoint and the way that portals work. The idea of being able to allow users to have various templates from which they could choose to make their profile page look, or even possibly give people the option to design their own page altogether was very appealing.

We did some early prototyping to see how far we could push some of the webpart plumbing that was newly available in Microsoft .NET 2.0. We were concerned that it would be extremely difficult to figure out how to get something workable to provide the plumbing we were going to need. As it turned out, we had a couple of parts that fit very well together. We had a custom built content management system that is database driven. We tossed a few web part zones into an ASPX page with a master page with multiview to support display and edit mode. The edit mode allows webparts to be pulled from the webpart catalog and stored on the page. The structure was then stored in our CMS. The entire blob could then be retrieved that had the page structure set with webpart placement. The webparts would then render. This set the stage for us to provide different templates by loading the page in edit mode, setting up the webparts differently, and then saving the metadata of the page layout off in the database. At runtime we then retrieve the desired page template for display. And a flexible and dynamically set system is born!

From the results to date, I would definitely call it an early success. We launched the myColts.net site back in mid-June. Since then we've added another profile layout which consists of nothing more than a page template with the webparts laid out differently on the page. Users have the option to set which layout they want, which is nothing more than storing the ID of the page to display with the chosen layout. Additionally, just today, I'm happy to announce the launch of myIndianaFootball.net. myIndianaFootball is a site that is built on the myColts.net technology to provide a site dedicated to Indiana High School football. The beautiful part about it is that it really isn't much more than a different theme applied to a set of webparts built to provide specific functionality for the high school site. It is running in the exact same shell as myColts.net, so there is an ultimate amount of code reuse in the core site.

Can you believe there are still places in this world where an always-on connection to the internet is not a given?

I was recently working on a project for a company that was in the process of finding a new system for their mining operations. This system would need to support employees in their field offices as well as management in their headquarters. As they were going through the selection process, they quickly eliminated systems based on their performance. The systems performed very well in “demo-in-a-box” situations. However, when a pilot was attempted at one of the field offices, network issues made all of the “web-based” contenders nearly unusable. Beyond terrible response time, the network would just plain drop off at random points during the day.

The easy answer that came to mind was: FIX THE NETWORK! After several meetings with business and IT senior leadership, it became clear that this easy answer was not feasible due to the dynamics of the organization. Why would IT improve the network for one business unit?

The next alternative that came to mind was: REPLICATION! Why not put a server at each of the field offices and replicate? Although much marketing material is written around replication by the major relational database vendors, basic concerns still need to be addressed such as uniqueness and conflict resolution. If the same item is edited by two users at different locations, is it ok for one of those users to lose their changes? Should a system administrator manually resolve this conflict? How quickly should users in other offices see changes? How do you assign a unique “business-relevant” identifier to an item?

We decided to go down the replication path. A master database was implemented at headquarters and slave databases at each of the field offices. We started with a replication period every 24 hours but quickly changed that to every 5 minutes when we discovered the importance of sharing the latest, greatest data with all sites. We currently have automatic conflict resolution configured so that the master database always wins in case of a conflict. Combined with the 5 minute replication, we’re seeing very few conflicts!

The key to a successful replication solution is ensuring these technical concerns are evaluated in light of the business requirements and expected customer experience.

We were preparing for a semi-public beta of the myColts.net site.  Our focus is on trying to release as high of a quality site from the start so that users can focus on the usability of the site and overall experience and less on the functionality side.  The Crowe development team and Colts testing team has spent hours and hours testing the site in various ways and scenarios.  However, I firmly believe anyone involved in the project eventually hits an educated capacity where they have used the site enough and are familiar with quarks and nuances enough it is easy to start over looking things.

Quality is a big focus for us at Crowe.  We firmly believe in trying to get the deliverable right the first time around, which can certainly be difficult in an iterative development process.  We talk a good game, but how real is it really?  I decided we could put it to the test to find out as well as help advance the testing of the myColts.net site.  Enter the contest.  I put some of my money at stake.  It is either going to the development team for a quality product or being doled out to those that find and report legit defects within the application.  The contest lasted two weeks with credit going to bug testers and payout at the end.  If there are remaining funds they get split up amongst the development team.  It is a bragging rights type of thing and a little friendly competition within my group. 

The end results were extremely positive.  Within less than a day of sending the message issuing the challenge there were a half dozen to a dozen folks registered within the site and trying it out.  I received a dozen legit errors and lots of questions, which are also great as they help expose areas where we may have been starting at and adopted to already where as a new user has not.  Got some good suggestions too which made their way in to the end site.  This very well could be a fan interest thing, or a novelty that applies here but isn't repeatable.  However, I've become a fan of community testing based on the results.

As we get closer to wrapping up the initial development on MyColts.net and releasing a public beta I was working out the design and implementation for the "Colts Cred" system.  The intent is to give away credibility points to people for various actions and help entice people to take desired action.  The first trick is to figure out the actions that will result in points being given, which was pretty straight forward.  I simply ran through the bulk of the functionality within the network and made a note of various actions (positive or negative) that could occur such as creating a blog entry, making a friend, commenting on a post of some sort, uploading photos, etc.  I then assigned an arbitrary positive or negative point value to each action knowing they are going to change.

Then came the fun part.  Working out a design that allows us to give away points in a way that won't chew up performance or add a ton of storage requirements.  Additionally allowing the point values for actions to be point in time so that certain actions could be promoted through "double points" type offers.  The solution to changing point values is pretty straight forward, which is just a name and value pair table.  For each action there is a name assigned to it and a point value assigned in the table to that name.  So when an action is taken the lookup is done based on the action name and points can be awarded or taken away based on the positive or negative value in the database.

The first inclination for the actual points was to just do an aggregate number that has total points and would be credited or negated with any action.  That certainly satisfies performance by having a rollup number available for use at any time for any user, but testing would be next to impossible to verify accuracy.  So we settled on a hybrid.  There is an audit table that has a record of the profile, action, points given or taken away at the time, and a flag indicating if the points have been applied to the running total or not.  There is a SQL Integration Services job that will periodically rollup the additional points, set the applied flag on audit records, and then update the aggregate total.  We can baseline the audit trail at any point if it becomes too big by recording the point value for each profile at the point the audit trail was cleared and then capturing the audit trail from that baseline.

The additional benefit of all of this is the work was all pushed to the database without resulting in extra calls from the web servers.  We simply added a stored procedure call to the end of most of the insert or update stored procedures, which took care of writing entries into the audit trail. 

Now at some point I suppose we'll have to work out what exactly you can do with the points, but we've already had quite the competition in the testing team to see who shows up in the top of the Colts Cred list!

One of the more complicated aspects of MyColts.net to implement is what we're calling friendlist security.  It allows you to control different views of your content and profile based on groupings you setup.  It gives you full control over who can see what information that you publish.  Thus, if your boss were to checkout your mycolts.net profile he may indeed see a completely different version of your content than your friends.

To make this work we started with a role based security system controlled by each user.  It is a classic create groups, put users into groups, and then assign permission to the groups scenario driven by each user.  Different components of the system (calendars, blogs, comments, photos) etc then plugin to the role based security system to govern the overall content and who is allowed to see it.  The UI relies on knowing the friendlist assignments by the content creator and comparing it to the intersection of assignments given to the current person viewing the content.

The implementation involves a single Permission entity.  That permission entity has a globally unique identifier (GUID) on it.  The GUID is stored along with the entity that owns the permission such as a blog or photo gallery.  When the permissions are edited the GUID is then joined to the permission list table for all of the profiles, friendlists, and groups that have been assigned rights to that content.  When there are no permissions assigned we set a bit flag indicating public permissions so that the rest of the permission checks can just be avoided if it is considered public content.  Since it is a social networking site we decided to use least restrictive rather than most restrictive on default permissions to try and encourage folks to share content.  Since all permissions are stored in a single set of tables we have a single control that can be used to edit the permissions on any entity type.  You simply bind the control to the permission GUID and it will allow you to edit and maintain the permission assignments and persist them back to the database.

There are a couple of joins required to check permissions on any entity, but so far the dynamics and flexibility of it have off set any performance issues.  Not to mention we've indexed the heck out of it.  It has proven quite a challenge to test.  There are individual, friendlist, and group assignments to consider.  Any number of times we've been convinced we found  a problem only to find that a user was a member of a group we didn't realize had permissions that was granting them access.  I'd heavily encourage anyone trying to build something similar to have some very explicit and detailed test cases to save yourself hours of headache and chasing phantom problems.

For the design process on MyColts.net we've been using a story board type design process that flows something like this:

• Upfront discussion about the site area / page and what functionality and purpose it will serve 
• Build a wireframe of the site area / page 
• List of functionality that the wireframe will support 
• Detailed graphic layout with example functionality 
• Construction of the item 
• Refactor based on feedback and use once it is plugged into the actual site 

The drawback to the process is the duration.  It can be tedious and time consuming to draw up the screen mockups for the site and adds time to the overall project.  It may have been a different experience if we had a dedicated designer, but unfortunately the on field success of the Colts has prevented that (a sacrifice we've all happily made).  Additionally, it has proven a challenge to be consistent with colors, styles, and behaviors across the screen designs.  There are new items that can often pop-in unannounced.

However, this has proven to be a pretty effective design process for us throughout the duration of the project.  We've had some good discussions around various pages and topics and found the upfront discussions help make sure concepts are better thought out up front.  It is a pretty visual process, which seems to especially benefit the programming team when it comes down to the actual construction and being able to implement the vision.  Folks that aren't naturally graphic inclined have a model to make the functionality jump from the start.  It's worked far better than detailed program specs and is a process I'd recommend considering.  As tools such as Microsoft Expression become more widely adopted perhaps adding more visual elements (right brained) to a classic programming (left brained) lifecycle will make story boarding even more practical and widely adopted.  I'd love to hear if this has worked for others.