My PhotoBruno Brás Silva

09, April 1986

info@brunosilva.net

Sintra Lisbon Portugal

Category: Open Source

HTTP Request and HTML Parsing in .NET

Posted on 7:26pm 5/17/2008 by Bruno Silva in .NET, Open Source, Programming, Utilities, Web

I’ve been working in college assignment which goal is to integrate several information systems. One of those is a website where we can see the flight schedules from/to Portuguese airports.

The website URL is http://www.innovata-llc.com/ana/default.asp. Me and a colleague of mine had to write a web service that was used by BizTalk as a wrapper of this web site’s form. It was supposed to send an HTTP request passing some request parameters using POST parameters, and to parse the HTML response in order to extract the flight list for a given destination city and departure date.

First of all lets see the code we used to get the response.

Uri address = new Uri(requestURL);
HttpWebRequest request = WebRequest.Create(address) as HttpWebRequest;
request.Method = “POST”;
request.ContentType = “application/x-www-form-urlencoded”;
StringBuilder data = new StringBuilder();

data.Append(“DPT_Date=” + “17-05-2008″);
data.Append(“&RET_Date=” + “20-05-2008″);
data.Append(“&dpt_station=” + “LIS”);
data.Append(“&arv_station=” + “LHR”);
data.Append(“&non_stops=” + “on”);

// Create a byte array of the data we want to send
byte[] byteData = UTF8Encoding.UTF8.GetBytes(data.ToString());
// Set the content length in the request headers
request.ContentLength = byteData.Length;
// Write data
using (Stream postStream = request.GetRequestStream())
{
postStream.Write(byteData, 0, byteData.Length);
}

// Get response
String htmlResponse;
using (HttpWebResponse response = request.GetResponse() as HttpWebResponse)
{
// Get the response stream
StreamReader reader = new StreamReader(response.GetResponseStream());
// Get the response string
htmlResponse = reader.ReadToEnd();
}

After getting the response content is time to use a nice HTML parser helper library. It is called HTML Agility Pack and it is open-source. You can find it at Codeplex.com/htmlagilitypack.

Now it is time to create an HtmlDocument (a class which ships with HTML Agility Pack) and load the response into this new instance. //Load HTML as XHTML
HtmlDocument htmlDoc = new HtmlDocument();
htmlDoc.LoadHtml(htmlResponse);

After looking to the HTML source code, we got the right Xpath expressions to extract the flight list.
//Get flight lines
HtmlNodeCollection flights = htmlDoc.DocumentNode.SelectNodes(
“//body/div/table[3]/tr[position()>=4 and position()<last()-1]“);

foreach (HtmlNode flight in flights)
{
//Get attributes
string departureTime = flight.ChildNodes[1].FirstChild.InnerText;
string arrivalTime = flight.ChildNodes[3].FirstChild.InnerText;
//Do some stuff …
}

That’s almost it. The next step was adding the extracted information into some data structures, and returning that data. But these details are out of this post scope.

Vertigo: Video.Show

Posted on 9:52pm 2/12/2008 by Bruno Silva in Open Source, Programming, Silverlight, Web

Some time ago I talked about Slide.Show. A slide show web application made with Silverlight.

Now the same company has published Video.Show. It is an open-source video sharing portal system based on Silverlight.

Video.Show

Learn more about features in their website. The source code is available at Codeplex. Unfortunately I think that Video.Show it much more unstable than Slide.Show. If you interact with it before the whole page is loaded, you get some Javascript errors. The live demo is a little slow too.

Video.Show

Sharp Develop

Posted on 11:12am 1/30/2008 by Bruno Silva in .NET, Open Source, Programming, Software

SharpDevelop

#develop (short for SharpDevelop) is a free and open-source IDE for C#, VB.NET and Boo programming language projects on Microsoft’s .NET platform. Of course that Microsoft Visual Studio Express Editions are freeware and are better that this IDE, but with #develop you get a chance to take a look a the source code and who knows, adapt it to develop in another language. #develop in written in C#.

SharpDevelop

The list of main features can be found in the project’s community website.

Vertigo: Slide.Show

Posted on 12:51pm 1/28/2008 by Bruno Silva in Freeware, Open Source, Programming, Silverlight, Web

Some time ago I’ve found an nice Silverlight 1.0 application which is now part of Silverlight.net Showcase. It is an open source configurable slide show application. The configuration is made by using XML or inline javacript (using JSON - JavaScript Object Notation).

Vertigo: Slide.Show

Vertigo: Slide.Show

See the live demo in Vertigo’s website. I had some issues while trying it on Firefox. The problem was Silverlight 1.1 September Alpha Refresh. So, in Internet Explorer it runs well with the 1.1 version, but to run it on Firefox you must have the stable 1.0 version.

This project is also hosted in Codeplex.

Wordpress PDA Plugin

Posted on 9:29pm 1/27/2008 by Bruno Silva in Freeware, Mobility, Open Source, Software, Web

Since I’ve created this blog I’ve been using a nice Wordpress plugin to allow a nice user experience while visiting this website with a mobile device. It is called Wordpress PDA Plugin and it’s installation is plug-and-play. Since this is a blog, and probably most of the readers just subscribe my RSS feed into some reader or aggregator, this plugin isn’t essential, but is still pretty nice.

These are some screenshots from my website mobile version.

Wordpress PDA Plugin Wordpress PDA Plugin

As you can see, in the homepage it removes all the images and shows only part of the text, to optimize performance. By entering into a entry details you can read the whole content, including images.

Songbird Media Player

Posted on 6:07pm 1/05/2008 by Bruno Silva in Freeware, Multimedia, Open Source, Software

Yesterday I took a look at Songbird Media Player. I must admit that I was very excited as a read about this open source media player. why? Because it has many things that look like my favorite web browser: Firefox! :)

The screenshot bellow shows the default look & feel of this application that looks a lot like iTunes. I like a lot the library navigation. You can filter the musics by year, artist and album in a simple way.

Songbird - Screenshot

How about Firefox similarities that I talked about? Here it goes. It is “developers friendly”. Has themes and add-ons support like Firefox. It allows tabbed browsing with a Firefox-based internal browser. As you can see in the screenshot below some interface elements like options and Add-ons menu are similar to the analogous ones in Firefox. Almost a copy of them, which can be nice while getting used to Songbird interface. This project is powered by Mozilla (which explains a lot :D ).

Songbird - Screenshot

The screenshot below which shows a album browser isn’t a built-in feature. It is a Java-based add-in which I installed. Looks nice but has some issues about resizing, and hangs a little bit too… You can navigate through the add-in list in here.

Songbird - Screenshot

My deception was when I played some songs… Songbird takes a lot of memory and CPU usage in order to work.

Songbird - Process details

I’ve tried iTunes 7.5 and tough it uses an considerable amount of memory it almost “doesn’t require” CPU time. I must confess that this memory usage of 47Mb doesn’t bother me. 2Gb of RAM is enough to handle it :) And I’m used to Firefox memory usage as you can also see in the screenshot.

Songbird - Process details

With Windows Media Player 11 I had some great results about both CPU and memory. It is the least I can expect with a media player built-in into the operating system.

Songbird - Process details

I must confess that I don’t use media players that often, and when I use it’s just to listen to music and leave the player running in background. Songbird seems promising, but for now I won’t use it because it doesn’t fit my needs (low memory, low CPU,shuffle and playing!). Although my experience wasn’t good, don’t stay stick to what I have written. Install it and give it a try!

© Bruno Silva | Powered by Wordpress