Image Manipulation and Storage

By Colin Nicholls
September 2005 (revised July 2006)

Briefing

OK, listen up. The situation is this: You have a network share full of periodically updated, high-resolution JPG images - stop giggling and get your mind out of the gutter, son, it's product artwork, not porn - and you need to write a utility that puts a reduced-in-size version of selected images into a SQL Server database so that they can be subsequently queried and retrieved for display in an ASP.NET application. The database will need to be refreshed on a periodic basis. Your development platform will be C# .NET. You have limited time, people, so let's FOCUS!

Planning

You're going to divide the utility application into two components: a library DLL containing the engine code; and a console EXE that leverages the engine. Remember console applications? They rock. Maybe in a future article we'll look at writing a Windows Forms interface to the engine DLL, so keeping these components independent is a good idea.

Overview

The console application - PictureGrabber - is going to need to:

• Retrieve configuration settings;
• communicate with the engine code;
• write error and progress feedback to the console.

The engine dll - PgEngine - is going to need to:

• understand job configuration parameters;
• obtain a list of image files in the file system that match specified filename masks;
• read in each image file;
• resize a jpg image;
• connect to a specified SQL Server database;
• save image data to a specified SQL Server table with a known schema.

These two components are going interact in the following way: The console application will instantiate a Job Configuration class, initialise its parameter values by reading from the app.config file, then instantiate an instance of the engine, and ask it to process the job configuration.

Better get started.

Creating the projects in Visual Studio

Open the Visual Studio edition of your choice [1]. Select File->New Project and select the Console Application template. Enter a project name of "PictureGrabber", and press the OK button.

Visual Studio has apparently created a "solution" containing our new project for you. How thoughtful. I found the concept of Solutions to be initially irritating, but in fact, they are a good idea, because you're  building two separate but related components, each with their own project. But do you have any idea where in your local file system it created this solution? Neither do I. Highlight the root Solution node in the Solution Explorer, then choose Save PictureGrabber.sln as... from the application File menu. (figure)

Enter the project's source location of your choosing. [2]

Now let's create the second project in your Solution. Select File->New Project. This time you want to select the Class Library template and enter a name of 'PgEngine'. At least this time the default location should be correct...

Starting with the skeleton of a console application

I have a rule that each source file in my projects contains just one class, and the source file is named after the class it defines. However I break this rule almost all the time. By default, Visual Studio has created a single source file called program.cs for our console application. Rename it to PgConsole.cs, because this is the name of the class we are about to implement. Enter the following code into the editor:

using System;

namespace PictureGrabber
{
    class PgConsole
    {
        static void Main(string[] args)
        { 

            Console.WriteLine("---------------------------------");
            Console.WriteLine("PictureGrabber v.1.0");
            Console.WriteLine("CLR v." + System.Environment.Version.ToString());
            Console.WriteLine();

We'll come back to this point and put more more code in here shortly. For now, let's just finish up the skeleton:

            Console.WriteLine();
            Console.WriteLine("Completed.");

        }
    }
}

This is the first (and last) time I'm going to remind you to save your work.

Job Parameters

Seeing as I've been through the process of completely implementing this utility, I'll save you some time and tell you that the PictureGrabber engine is going to need the following information:

• a list of file system paths to scan for image files;
• a list of filename masks to filter out unwanted file types;
• several image shrinking parameters;
• a connection string for the destination database;
• the table to store the size-reduced images in;
• whether to completely replace or update the data in the table.

Let's implement a class in the PgEngine project to encapsulate this configuration. Right click on the class1.cs node in the solution and rename it to PgConfig.cs. Start coding:

using System;
using System.Collections.Specialized;

namespace PictureGrabber
{

    public class PgConfig
    {

Use regular fields [3] to hold the parameters. Use a type of StringCollection to hold each list of source directories and file masks:

        public StringCollection sourceDirectories;
        public StringCollection fileMasks;

        public int      imageQuality;
        public int      imageWidthMax;
        public int      imageHeightMax;

        public string   connectionString;
        public string   destinationTable;
        public bool     truncateTable;

It might be useful to have an errorMessage property:

        public string   errorMessage;

It's also a good idea to implement a constructor method to initialise the properties:

        public PgConfig()
        {
            this.sourceDirectories = new StringCollection();
            this.fileMasks         = new StringCollection();

            this.truncateTable    = false;
            this.destinationTable = "artwork";
            this.connectionString = "";

            this.imageHeightMax   = 450 ;
            this.imageWidthMax    = 450 ;
            this.imageQuality     = 80 ;

            this.errorMessage     = "";
        }
    }
}

The application configuration file

It's tempting to write an old-school method of the PgConfig class that reads the job parameters in from an INI file. But .NET has a shiny new method of storing initialization settings: Application configuration files. They are, of course, in XML format.

Select the PictureGrabber project node in the Solution Explorer and select Add New Item... from the Project menu. Choose the 'Application Configuration File' template (figure) and press the Add button. Visual Studio will create a new configuration file from the template and open it in the editor. Visual Studio has a whacked-out mechanism of copying this file out alongside the .exe as part of the build process. And I quote from the help file: When you build your project, the development environment automatically creates a copy of your app.config file, changes its file name so that it has the same file name as your executable, and then moves the new .config file in the bin directory. You'll see how this works out in practice shortly.

Let's create a possible configuration file that describes the required job parameters:

<configuration>

    <dataManager
        connectionString="Data Source=DB_SERVER; Connect Timeout=5; Initial Catalog=Products; User Id=svc; Password=svc"
        table="artwork"
        truncate="Yes" />

    <imageToolkit
        quality="80"
        maxWidth="450"
        maxHeight="450" />

    <fileMasks>
        <item>595*.jpg</item>
        <item>692*.jpg</item>
        <item>733*.jpg</item>
    </fileMasks>

    <sourceDirectories>
        <item>C:\product_art\1999</item>
        <item>C:\product_art\2000</item>
    </sourceDirectories>

</configuration>

Configuration section handlers

Your application needs to know how to handle each child node, or section, of the configuration file. Lucky for us, the .NET framework contains several pre-built configuration section handlers that you can use right off the bat. For example, a single node with a list of name-pair attributes (such as the dataManager and imageToolkit nodes, above) can be read by class called SingleTagSectionHandler.

You assign a handler class to each configuration section through an additional child node of the config file, configSections:

    <configSections>
        <section name="dataManager"
                 type="System.Configuration.SingleTagSectionHandler" />

        <section name="imageToolkit"
                 type="System.Configuration.SingleTagSectionHandler" />

What about the two lists of file masks and source directories? Well, it's possible that a pre-built handler class for these already exists somewhere in the .NET framework, but I couldn't find it. Which is a great excuse to learn how to write your own, custom section handler!

Writing a custom section handler

Section handler classes must implement the IConfigurationSectionHandler interface. You're going to write a class that implements this interface called StringCollectionSectionHandler in the PgConfig.cs file. First, you should add a reference to the System.Xml namespace at the top of the file:

using System;
using System.Collections.Specialized;
using System.Xml;

Now implement the class. Just put it in the same file, directly below the PgConfig class definition:

namespace PictureGrabber
{
    public class PgConfig [...]

    public class StringCollectionSectionHandler : System.Configuration.IConfigurationSectionHandler
    {
        public StringCollectionSectionHandler() { }

Now here's the signature of the sole method in the IConfigurationSectionHandler interface: Create().

        public object Create( object parent, object configContext, System.Xml.XmlNode section )
        {

It is straightforward to process the XML node and return a populated StringCollection object:

            StringCollection itemList = new StringCollection();

            foreach (XmlNode child in section.ChildNodes)
            {
                if (child.NodeType == XmlNodeType.Element)
                {
                    itemList.Add(child.InnerText);
                }
            }
            return itemList;
        }
    }
}

Completing the app.config

Now that you have your custom section handler, you can complete the rest of the app.config file, attaching the handler class to the fileMasks and sourceDirectories nodes:

        <section name="fileMasks"
                 type="PictureGrabber.StringCollectionSectionHandler,PgEngine" />

        <section name="sourceDirectories"
                 type="PictureGrabber.StringCollectionSectionHandler,PgEngine" />

    </configSections>

Note the additional qualifier on the type attribute value. Unlike the built-in section handler classes, you need to specify what assembly contains the custom handler class. In our case, that is PgEngine.dll.

Completing the config class

PgConfig :: LoadFromAppConfig()

You should now implement a method of the configuration class PgConfig that initializes its fields from the config file. Call it LoadFromAppConfig:

        public bool LoadFromAppConfig()
        {
            try
            {

For the SingleTagSectionHandler, the syntax is pretty funky. You get passed back an attribute name key-based list (or IDictionary) of objects, which you then need to cast into strings:

                System.Collections.IDictionary param;
                param = (System.Collections.IDictionary) System.Configuration.ConfigurationManager.GetSection("tableManager");
                this.connectionString = (string)param["connectionString"];
                this.destinationTable = (string)param["table"];

Because the attribute values in the config file all come back as strings, you need to get more creative to initialize your boolean and integer fields:

                string cTruncate = (string)param["truncate"];
                this.truncateTable = cTruncate.ToUpper().Equals("YES");

                param = (System.Collections.IDictionary) System.Configuration.ConfigurationManager.GetSection("imageToolkit");
                this.imageHeightMax = short.Parse((string)param["maxHeight"]);
                this.imageWidthMax  = short.Parse((string)param["maxWidth"]);
                this.imageQuality   = short.Parse((string)param["quality"]);

In contrast, your custom section handler code is trivial:

                this.fileMasks =         (StringCollection) System.Configuration.ConfigurationManager.GetSection("fileMasks");
                this.sourceDirectories = (StringCollection) System.Configuration.ConfigurationManager.GetSection("sourceDirectories");

                return true;
            }
            catch (Exception eee)
            {
                this.errorMessage = eee.Message;
                return false;
            }
        }

Note that you get to use the errorMessage property to report back if you encounter a problem reading the config file.

PgConfig :: DumpToConsole()

Finally, create a utility method, DumpToConsole,  that dumps the configuration parameters out to the console, for testing or debugging purposes:

        public void DumpToConsole()
        {
            Console.WriteLine("-------------------------------------------");
            Console.WriteLine("Current configuration values:");
            Console.WriteLine("-------------------------------------------");
            Console.WriteLine("Connection String = {0}", this.connectionString);
            Console.WriteLine("Destination Table = {0}", this.destinationTable);
            Console.WriteLine("Truncate = {0}",          this.truncateTable);
            Console.WriteLine("Image Quality = {0}",     this.imageQuality);
            Console.WriteLine("Image Max Width = {0}",   this.imageWidthMax);
            Console.WriteLine("Image Max Height = {0}",  this.imageHeightMax);
            Console.WriteLine("File masks:");
            foreach (string mask in this.fileMasks)
                Console.WriteLine("   " + mask);
            Console.WriteLine("Source directories:");
            foreach (string dir in this.sourceDirectories)
                Console.WriteLine("   " + dir);
            Console.WriteLine("-------------------------------------------");
        }

You're going to use this method to test your progress so far.

Plug in and test

At this point, you can go back to your main console application class definition that we wrote earlier in PgConsole.cs, and tell it to instantiate the configuration class:

            Console.WriteLine("---------------------------------");
            Console.WriteLine("PictureGrabber v.1.0");
            Console.WriteLine("CLR v." + System.Environment.Version.ToString());
            Console.WriteLine();

            // Create an instance of the job configuration class:

            PgConfig jobConfig;
            jobConfig = new PgConfig();

            // Initialise the configuration out of the app.config file:

            if( jobConfig.LoadFromAppConfig() )
            {
                //------------------------------------
                // Temporary: List the parameters:

                jobConfig.DumpToConsole();

                //------------------------------------
            }
            else
            {
                Console.WriteLine("Error: " + jobConfig.errorMessage );
            }

            Console.WriteLine("Press any key...");
            Console.ReadLine();

            Console.WriteLine();
            Console.WriteLine("Completed.");

If you now try to build the solution (which will compile both projects) you will find out that there is something else you need to do. You may encounter the following build error: The type or namespace name 'PgConfig' could not be found (are you missing a using directive or an assembly reference?).

Actually, if you typed this code into the Visual Studio editor manually, then you would have noticed that you weren't getting IntelliSense prompts for the PgConfig members. This is because - even though both class definitions have been declared to share the same namespace, and their projects are in the same Solution - Visual Studio hasn't figured out what we want to do. There's probably a good reason for this, but no worries - it's easy enough for you to join the dots yourself: Select the PictureGrabber project in the solution tree and select Add Reference... from the Project menu. In the Add Reference dialog box (figure), select the Projects tab and select the PgEngine project. Having applied this change, rebuilding the project should no longer cause those errors to be generated. You can see the PgEngine project reference show up in the PictureGrabber project tree.

After the build process has completed successfully, the PgEngine.dll, PictureGrabber.exe, and configuration file have been written out to the \bin\Release directory (figure), along with associated .pdb files [4].

If you're in Visual Studio, you can hit F5 to run the application in debug mode, or if you're in a retro frame of mind, you can open a command prompt window and run PictureGrabber.exe from there:

C:\work\...\PictureGrabber\bin\Release>picturegrabber
---------------------------------
PictureGrabber v.1.0
CLR v.2.0.50727.7

-------------------------------------------
Current configuration values:
-------------------------------------------
Connection String = Data Source=DB_SERVER; Connect Timeout=5; Initial Catalog=Products; User Id=svc; Password=svc
Destination Table = artwork
Truncate = True
Image Quality = 80
Image Max Width = 450
Image Max Height = 450
File masks:
   595*.jpg
   692*.jpg
   733*.jpg
Source directories:
   C:\product_art\1999
   C:\product_art\2000
-------------------------------------------
Press any key...

-------------------------------------------
Completed.

You should fix your bugs now, if you have any. I didn't.

Next Steps

In the next installment of this article series, we will look at creating the framework of a PgEngine class that knows how to process an instance of PgConfig. We'll also look at creating a file system tree crawler class that uses delegates. In future articles, we will design our artwork table in SQL Server; writing image processing code to manipulate the image files, and perhaps look at implementing an ASP.NET application that displays these images from the database.

You can download a zip file containing PgConsole.cs, PgConfig.cs, and app.config as described in this article here: part1_source.zip.

Footnotes:

[1]  I started writing this article using the August CTP edition of Visual C# Express 2005.  It didn't blow up in my face but I kept my flak goggles handy. I continued the project using the release version of Visual Studio 2005 Professional.

[2]  I store my projects in a separate directory for each client, in a share that gets backed up on a specific schedule.

[3] Yeah, the code nazis will tell you to use properties with get/set methods. You know there's a performance penalty associated with doing that all the time? Visual Studio 2005's refactoring tools make it dead easy to turn a field into a property at any time, and it won't break your API when you do. I recommend using fields unless you know for a fact you need to attach get/set code to them.

[4] .PDB files contain debugging information. You do not need to deploy these files on a production system. You may also see *.vhost.exe and *.vhost.exe.config files. These are new to Visual Studio 2005 and have to do with improved debugging performance. There is a good explanation of these files here: http://blogs.msdn.com/dtemp/archive/2004/09/09/215764.aspx.

Acknowledgements:

Thanks to everyone out there who - for whatever reason - decides to put articles and code example up on the web for people like me to learn from. I have been pretty lax about keep records of the various web sites I found through Google that explained some of this stuff, but I'll try to do better in the future.