Making your input forms more intelligent.

Online sample: Live Demo

There is a large number of applications that are related to entering data that is on paper, these applications always enter the documents from a scanner or import them from a data store and the data entry officer will start filling the forms, some of the forms even have auto OCR for certain areas of the document. Today I want to work on enabling some advance technics that is related to this subject, no OCR detail here (mostly OCR work well only with Western languages like English).

In my case I want my data entry officer to have an easy time entering his information to the form after getting a scan image from the scanner. I don’t want him to have the paper in hand; I don’t want him to switch from one window to another to copy parts of the document as Invoice Number, Owner Name & etc.

My Goal:

• I want the data entry person to see the image side by side with the fields he is entering and the application will focus automatically on the regions that he is filling, as for example, if he is entering information about the Invoice Number, I want the application to zoom on that part of the document, when the data entry person goes to the Owner Name, I need the application to focus and zoom on the Owner Name.

• I need it to work on asp.net pages (web application)
• I want to make it easy to be used on other applications (as a component for example)
• It must be free, no payments here or there.

Tools:

• VS2010/VS2008 for development with C#/VB
• A very cool control called Seadragon

Let’s start, making this a short document is hard so I will need to reference some other blogs that will help me on this subject.

Visit my first blog about this control (Using the Seadragon control with an Image library) it will help us make the control work with an image of our desire.

Open a web site project and download the seadragon-min.js, its available on the site.

Add the following to your page header:

<script type="text/javascript" src="Scripts/seadragon-min.js"></script>

<script type="text/javascript">

var viewer = null;

var Loc_x = .5;

var Loc_y = .5;

var loc_zoom = 2; 

function init() {

viewer = new Seadragon.Viewer("container");

viewer.openDzi("dzc_output.xml");

}

Seadragon.Utils.addEvent(window, "load", init);

function SetFocus(x,y,z) {

Loc_x = x;

Loc_y = y;

loc_zoom = z; 

FocusOn();

}

function FocusOn(event) {

Seadragon.Utils.cancelEvent(event); // don't process link

if (!viewer.isOpen()) { return; }

viewer.viewport.panTo(new Seadragon.Point(Loc_x, Loc_y));

viewer.viewport.zoomTo(loc_zoom);

viewer.viewport.ensureVisible();

}

Seadragon.Utils.addEvent(window, "load", init);

</script>              

Simply we are adding 3 variables, Loc_x & Loc_y for the location, and loc_zoom for the zoom factor.

Let’s add four text boxes simulating our data enter form side by side with the Seadragon:

<asp:ScriptManager ID="ScriptManager1" runat="server"/>        

<table>

<tr><td class="style2" rowspan="4"><div id="container"></div></td>

<td>

Inv. Number: <asp:TextBox ID="TextBox1" runat="server" />

</td></tr>

<tr><td> 

Inv. Owner Name:<asp:TextBox ID="TextBox2" runat="server"/>

</td></tr>

<tr><td>

Inv Date:<asp:TextBox ID="TextBox3" runat="server"/>

</td></tr>

<tr><td>

rest to home <asp:TextBox ID="TextBox4" runat="server"/>

</td></tr>

</table>

And in our page load event

If (Not IsPostBack) Then

TextBox1.Attributes.Add("Onfocus", "SetFocus(.7,.2,8);")

TextBox2.Attributes.Add("Onfocus", "SetFocus(.4,.5,2);")

TextBox3.Attributes.Add("Onfocus", "SetFocus(.3,.9,2);")

TextBox4.Attributes.Add("Onfocus", "SetFocus(.5,.5,1);")

End If

Each control is hocked with the onfocus event and specifying the location and the zoom factor needed.

In general the width of the document is always 1, the height depends on other factors. If you like to read more about the coordination system go here.

I prefer using zoom factor as 1,2,4,8,16,32 & etc

We can simply warp this in a control containing the viewer and some function to hock the control from the data entry fields.

The Identification  of the active points in the document are always related to the document type (so some database can be used here to focus on needed parts depending on the document type)

Code Formatter

here is a very useful link to format your code for publishing

http://www.manoli.net/csharpformat/format.aspx

Using the Seadragon control with an Image library

Seadragon is one great control in the Ajax Control Tool kit, if you didn’t see it yet, check it out here, the idea is that it gives you a very cool image viewer to you page with DeepZoom Technology. So high resolution images is not a problem anymore.

Alone the control does everything you need, but it forces you to have your image pre-rendered (with Deep Zoom Composer or any other tool) . So what if we need to make this control work with our image library without creating all Seadragon images (without thinking about maintaining them after updates and moving them to your production nor even the extra size). It’s a small challenge.

1. So after looking at the problem my approach is as follow:
   
   Create an Http Handler to handle the requests from the Seadragon Control
   
   
1. For IIS6 register the jpg extension to be handled by our handler
2. Add the needed configurations to our Web.config
3. Create our handler
2. Create the needed Seadragon images on the fly. Providing the needed part only to the control.
   
   1. Finding the needed level and which part the control need
   2. Getting the high resolution image, making some zooming and slicing
   3. Returning the result to the control by the handler.
   
   

Simple, that’s it. We don’t need anything else.

Before going in the details, I only wanted to say that this is only the first phase. For future I will try to make it work with PDF documents with some special zooming tricks. But this is not our problem so let’s go back.
For the first task (1.1) registering the jpg extension in IIS, that’s a simple task
Launch the Internet Services Manager tool, right click on Default Web Site, select Properties, go to Home Directory tab and press Configuration button. This will popup Application Configuration dialog. Click Add button and fill the Executable field with the path to the aspnet_isapi.dll file and fill .jpg in the Extension field. Leave the other fields as is, close the Application Configuration and Default Web Site Properties dialog boxes by pressing the OK button. (if you have IIS7 this step is not needed).

For the task (1.2) Add the needed configurations to our Web.config, add the following:
Add the following in the section of the web config

<httpHandlers>

<add verb="*" path="dzc_output_files/*.jpg" type="ImageHandler.HttpImageHandler,ImageHandler"/>

</httpHandlers>

So now requests coming for the control are captured by our handler, so let’s create it.
For the first task (1.c), Create our handler
using System;

using System;

using System.Web;

using System.IO;

using System.Drawing;

using System.Drawing.Imaging;

using System.Text.RegularExpressions;

using Injazat.DeepZoom;

namespace ImageHandler

{

public class HttpImageHandler : IHttpHandler

{

public void ProcessRequest(System.Web.HttpContext context)

{

DynamicDeepZoomImageGenerator DDZIG = new DynamicDeepZoomImageGenerator();

if (context.Request.PhysicalPath.IndexOf(DDZIG.Identifier) < 0)

{

//this is not for the deep zoom images 

context.Response.Clear();

context.Response.ContentType = getContentType(context.Request.PhysicalPath);

context.Response.WriteFile(context.Request.PhysicalPath);

context.Response.End();

}

else

{

context.Response.Clear();

context.Response.ContentType = getContentType(context.Request.PhysicalPath);

byte[] buffer = DDZIG.GeneratorImage(context.Request.PhysicalPath);

context.Response.OutputStream.Write(buffer, 0, buffer.Length);

context.Response.End();

}

}

public bool IsReusable

{

get

{

return false;

}

}

string getContentType(String path)

{

switch (Path.GetExtension(path))

{

case ".bmp": return "Image/bmp";

case ".gif": return "Image/gif";

case ".jpg": return "Image/jpeg";

case ".png": return "Image/png";

default: break;

}

return "";

}

}}

So we need now to create the needed Seadragon images on the fly
I created one class to handle that.

using System;

using System.Collections.Generic;

using System.Linq;

using System.Text;

using System.IO;

using System.Drawing;

using System.Drawing.Imaging;

using System.Text.RegularExpressions;

namespace Injazat.DeepZoom

{

public class DynamicDeepZoomImageGenerator

{

int Size = 250;

public string Identifier = "dzc_output_files";

public DynamicDeepZoomImageGenerator()

{

}

public byte[] GeneratorImage(String PhysicalPath)

{

// 1. find the orignal size of the doc

String path = @"C:\inetpub\wwwroot\testthumb\SampleDoc.jpg";

Bitmap imgIn = new Bitmap(path);

Point OriginalSize = new Point(imgIn.Width, imgIn.Height);

// 1.5 find number of zoom level avaliable

int ZoomMax = 0;

int X = OriginalSize.Y;

do { X /= 2; ZoomMax++; } while (X > 0);

ZoomMax = (int)Math.Ceiling(Math.Log(Math.Max(imgIn.Width, imgIn.Height), 2));

// 2. get the zoom level and array location

int ZoomLevel = 0;

Point ArrayLocation = new Point(0, 0);

String Pattren = Identifier + @"\\(?<ZoomLevel>[0-9]{1,2})\\(?<XXX>[0-9]{1,2})_(?<YYY>[0-9]{1,2}).jpg";

// use 'x' modifier to ignore comments

Regex r = new Regex(Pattren);

// get the list of group numbers

int[] gnums = r.GetGroupNumbers();

// get first match

Match m = r.Match(PhysicalPath);

if (m.Success)

{

ZoomLevel = int.Parse(m.Groups["ZoomLevel"].Value);

ArrayLocation.X = int.Parse(m.Groups["XXX"].Value);

ArrayLocation.Y = int.Parse(m.Groups["YYY"].Value);

}

// 3. return the needed part depending on the req.

//find the scalling 

int Scale;

if (ZoomMax > ZoomLevel)

{ Scale = (int)Math.Pow(2, (ZoomMax - ZoomLevel)); }

else { Scale = 1; }

// resize the image to the scale

Bitmap imgOut = ResizedImage(imgIn, Scale);

// get only the needed portion of the image

imgOut = GetRegionImage(imgOut, ArrayLocation, Size);

return Image2BateArray(imgOut);

}

byte[] Image2BateArray(Bitmap imgIn)

{

System.IO.MemoryStream outStream = new System.IO.MemoryStream();

imgIn.Save(outStream, ImageFormat.Jpeg);

return outStream.ToArray();

}

public Bitmap ResizedImage(Bitmap imgIn, int Scale)

{

if (Scale == 0) Scale = 1;

int NewWidth = (int)(1.0F * imgIn.Width / Scale);

int NewHeight = (int)(1.0F * imgIn.Height / Scale);

if (NewWidth == 0) NewWidth = 1;

if (NewHeight == 0) NewHeight = 1;

Size NewSize = new System.Drawing.Size(NewWidth, NewHeight);

Bitmap imgOut = new Bitmap(imgIn, NewSize);

return imgOut;

}

Bitmap GetRegionImage(Bitmap imgIn, Point ArrayLocation, int RegionSize)

{

Point TopLeft = new Point(ArrayLocation.X * RegionSize, ArrayLocation.Y * RegionSize);

Size NewSize = new Size(RegionSize, RegionSize);

if ((TopLeft.X + RegionSize) > imgIn.Width) NewSize.Width = imgIn.Width - TopLeft.X;

if ((TopLeft.Y + RegionSize) > imgIn.Height) NewSize.Height = imgIn.Height - TopLeft.Y;

Rectangle SourceRegion = new Rectangle(TopLeft, NewSize);

Bitmap imgOut = new Bitmap(RegionSize, RegionSize);

Graphics g = Graphics.FromImage(imgOut);

g.Clear(Color.White);

g.DrawImage(imgIn, new Rectangle(0, 0, NewSize.Width, NewSize.Height), SourceRegion, GraphicsUnit.Pixel);

g.Dispose();

return imgOut;

}

void DrawOnImage(ref Bitmap imgIn, String Text)

{

Graphics g = Graphics.FromImage(imgIn);

Font Fo = new Font("Arial", 12);

Rectangle Clip = new Rectangle(0, 0, imgIn.Width, imgIn.Height);

g.DrawString(Text, Fo, Brushes.Blue, Clip);

g.DrawEllipse(Pens.Red, Clip);

Fo.Dispose();

g.Dispose();

}

ImageFormat getImageFormat(String path)

{

switch (Path.GetExtension(path))

{

case ".bmp": return ImageFormat.Bmp;

case ".gif": return ImageFormat.Gif;

case ".jpg": return ImageFormat.Jpeg;

case ".png": return ImageFormat.Png;

default: break;

}

return ImageFormat.Jpeg;

}

string getContentType(String path)

{

switch (Path.GetExtension(path))

{

case ".bmp": return "Image/bmp";

case ".gif": return "Image/gif";

case ".jpg": return "Image/jpeg";

case ".png": return "Image/png";

default: break;

}

return "";

}

}

}

So finalizing our work
Create an xml file named “dzc_output.xml”, that will contain:

<?xml version="1.0" encoding="utf-8"?>

<Image TileSize="256" Overlap="0" Format="jpg" ServerFormat="Default" xmlns="http://schemas.microsoft.com/deepzoom/2009">

<Size Width="754" Height="948" /></Image>

Add an web site with a default.aspx page and include the following:

<asp:ScriptManager ID="ScriptManager1" runat="server">

</asp:ScriptManager>

<cc1:Seadragon ID="Seadragon1" runat="server" SourceUrl="~/dzc_output.xml" 

BorderColor="#660033" BorderStyle="Double" BorderWidth="4px" 

Height="240px" AutoHideControls="False" MaxZoomPixelRatio="10" Width="200px">

</cc1:Seadragon>

In my project I added the handler and the DynamicDeepZoomImageGenerator class to a separate project and referenced it.
That’s it.
You can see from the code that the image loaded is one image , in my next post I will add the needed code not only to change the image, but to extract on from a PDF and display it, and focus on some important locations too.  (It depends on the time :) )


Related Links:

• AJAX Toolkit
• Seadragon control by visiting the Microsoft Live Labs website.

Using Google Translation service from Net code

This has been out there for a long time, anyhow I found it know and I think it will be very cool to have our application translate on the fly whatever text we need to any language we desire.

If you think you don’t need translation or that its not so good (need human translation help) you are not so right.

For example you can translate names, and subjects from one language to another on the fly, belive me it will help a lot if you are working with international appcliations

So let’s start, Google provides these links as reference to its translation service

http://code.google.com/apis/ajaxlanguage/documentation/

So after a quick review, we can consume it by calling the service as an HttpWebRequest, and get the result back.

So for example:

http://ajax.googleapis.com/ajax/services/language/translate?v=1.0&q=Ahmed Khaled&langpair=en|Ar

the service will reply back with the following:

{"responseData": {"translatedText":"احمد خالد"}, "responseDetails": null, "responseStatus": 200}

Ok, I will give a simple VB example here (to warp the result back):

Public Shared Function Translate(

ByVal Text As String, ByVal FromLanguage As String

, ByVal ToLanguage As String) As String

'Create the HttpWebRequest object

Dim URL As String = String.Format(GoogleServiceURL, Text, FromLanguage, ToLanguage)

Dim req As HttpWebRequest = WebRequest.Create(URL)

'Set the timeout to 2 second (or 1,000 milliseconds)

req.Timeout = 2000

''Create the proxy class instance if needed

'Dim prxy As New WebProxy("http://xxx.xxx.xxx.xxx:xxxx")

''Specify that the HttpWebRequest should use the proxy server

'req.Proxy = prxy

Try

'Get the data as an HttpWebResponse object

Dim resp As HttpWebResponse = req.GetResponse()

'Convert the data into a string (assumes that you are requesting text)

Dim sr As New StreamReader(resp.GetResponseStream())

Dim results As String = sr.ReadToEnd()

sr.Close()

Dim GRD As New GoogleResponseData(results)

' ... work with results ...

Return GRD.TranslatedText

Catch wex As WebException

Return "Error in Translation: " + wex.Message

'Something went wrong in the HTTP request!

End Try

End Function

The GoogleResponseData is only a simple class that will separate the returning string to some object you can use,

Private Class GoogleResponseData

Private _TranslatedText As String

Private _ResponseDetails As String

Private _ResponseStatus As String

Public ReadOnly Property TranslatedText() As String

Get

Return _TranslatedText

End Get

End Property

Public ReadOnly Property ResponseDetails() As String

Get

Return _ResponseDetails

End Get

End Property

Public ReadOnly Property ResponseStatus() As String

Get

Return _ResponseStatus

End Get

End Property

Public Sub New(ByVal Result As String)

'Result format

'{"responseData": {"translatedText":"Hello"}, "responseDetails": null, "responseStatus": 200}

Result = Result.Replace("""", "")

Dim rx As New Regex("{responseData: {translatedText:(?.*)}, responseDetails: (?.*), responseStatus: (?.*)}", RegexOptions.Singleline)

Dim mc As MatchCollection = rx.Matches(Result)

If mc.Count = 1 Then

_TranslatedText = mc(0).Groups(1).Value

_ResponseDetails = mc(0).Groups(2).Value

_ResponseStatus = mc(0).Groups(3).Value

End If

End Sub

End Class

I love google services.. J