I love Mozy backup, but..

by Ryan 3. February 2010 13:51

WTF?

The machine was up for 10 days and the mozybackup service had consumed half of my available RAM.  Restarting the service released the RAM, as expected, but that seems a bit extreme.

 

In all fairness, I was 3 patch-releases behind, though there's no auto-update feature or notification in Mozy (yet?).  I upgraded to the latest version and will keep an eye on it.  Since Mozy doesn't post release notes or a changelog, I can only hope for the best.

 

EDIT: Updating to the latest version solved the problem.  The mozybackup process is appropriately consuming 33MB after 14 days uptime.

 

An Approach to Multiple Browser Session Detection

by Ryan 2. February 2010 10:24
Disclaimer: This is more academic than anything. Ideally you'd want to avoid a design that is flawed with respect to multiple sessions. Careful consideration for this, up front, should be taken to avoid needing a band-aid such as this.


I'm sure we've all worked on, or are aware of web applications (especially in the enterprise) that have tightly bound themselves to the server session. In these cases, it's possible that the session will become corrupted if more than one browser session is open and using the same server session cookie.

I've examined all of the options and found the best way to move forward would be to discourage the use of multiple browser sessions that share a server session cookie. This is only really a problem when a user executes "New Window - Ctrl+N" in IE or the equivalent of "duplicate tab" in other browsers. Essentially we end up with two active browser sessions sharing the same cookies.

So, to discourage this, as it will likely be inadvertent, some kind of warning system should be in place to prevent this behavior. Now, the underlying code should do plenty of concurrency checking to ensure data integrity, but there are some situations where issues can result from viewstate corruption.

A solution that I came to, after finding that the general answer is "it's impossible", was to rely on AJAX to send out "pings" and measure the time between. So, we have a general rule: we "ping" at a certain interval and if the delta between the last ping in the current ping is *less* than the ping duration, we know we have multiple active browser sessions on a single server session.

So, where Pf is ping frequency; Pc is current ping; and Pl is last ping, then we have an error when Pf > (Pc - Pl).

           p1    p2    p3    p4
TAB1 0-----|-----|-----|-----|---...
                 :     :     :
                 :  p1 :  p2 :  p3    p4
TAB2          0-----|-----|-----|-----|---...
     ^     ^     ^  ^  ^  ^  ^  ^
                  Deltas

----+---+------------
TAB | P |   Delta (Pc - Pl)
----+---+------------                 
 1  | 1 |   5
 1  | 2 |   5
 2  | 1 |   2.5 -Error
 1  | 3 |   2.5 -Error
 2  | 2 |   2.5 -Error

Now, if there is network congestion or other factors, then the delta will be greater than the frequency, ruling out false-positives.

We do have a problem if two tabs are open at the exact same momemnt. But, since the ping frequency is just the frequency at which the requests are made, and not a guaranteed elapsed time, we can assume that soon the two browser sessions will begin sliding out of sync.

In the example, I have the ping frequency set to every 5 seconds. If there are 100 simultaneous users then we're looking at ~20 requests/second for the ping Servlet/HttpModule. To minimize unnecessary network traffic I was thinking that the ping frequency would decay as time went on until a maximum of 20 pings/second was reached. This would amount to ~5 requests/second with 100 concurrent users. This is a trade-off, though, as it will cause a delay in detection. However, once detection occurs, the frequency resets to 5 pings/second until resolved. (These numbers are just as an example; they would vary a based on the environment)

To minimize concurrency and scalability issues, the last ping timestamp for the session should be kept in the session itself. This will allow any distributed session technology to maintain the availability of the session across JVMs or app domains without our ping service needing to be aware of it.

Brainstorming: Events

by Ryan 20. January 2010 17:28

While working on my side project, I’ve spent a lot of time trying to work out the best design. Keep in mind that this is more of a hobby for me, so I’m not trying to rush to publish a prototype to get VC.

I’ve worked in the enterprise application development space for several years, so I’m plenty familiar with separations of concerns. I realized, however, that there’s no good solution for handling the ancillary, “can the current user do X” and “if the current user does X, then do Y”. I’ve addressed this issue a number of ways in the past, none of which was particularly elegant. For instance, I’ve used AOP for cross-cutting concerns, but I found it too hard to test and not quite explicit enough.

So, I realized that these “meta-domain” concerns really aren’t concerns for the domain objects themselves. For instance, the User entity doesn’t care if a link is displayed on the user profile page based on security. In this case we could have a security manager that deals with this, but what if this decision isn’t just based on security, but also user reputation, user history and user profile settings? Do I suddenly need to couple my view-model to all of these managers/services? Do I have to have all of these redundant dependencies for every page? What if I forget one? How do I easily test what will happen?

These are all questions that I felt lacked a good answer. As a response, I created a library that is capable of registering “events” and “event” handlers. These aren’t traditional events, but more like “domain events” in that their scope is the entire domain.

Now, sure, there are other ways of accomplishing this, but my particular set of requirements calls for multiple actions to take place when certain events are fired. I also need to know if I can execute a given event, which makes them more than just fire-and-forget.

I had several requirements for my design: it had to be easy to use and understand, it had to play well with an IoC container, it had to be testable and it had to be as type-safe as possible. What I came up with seems to have accomplished all of these, though I haven’t put it to use just yet.

Here’s a snippet from one of the tests that illustrates one of the use cases. The AddNew method is defined on the IMockEventHandler interface. This returns a ValidationResult indicating whether or not the AddNew event can be called in the given MockContext based on all IMockEventHandler objects.

   1:  var result = eventManager.For<IMockEventHandler>()
   2:                           .In( new MockContext() )
   3:                           .Where( x => x.GetType().FullName.Contains( "Mock" ) )
   4:                           .Where( x => x.EventSourceType == null )
   5:                           .Validate( x => x.AddNew( "s" ) );

Typically the Where criteria won’t exist, they’re just there for flexibility.

So from the code snippet you can see some of the opportunities it presents. Say I have the following:
   1:  public interface IOrder {
   2:      void ShipOrder( string orderNumber );
   3:  }

   1:  public interface IOrderEventHandler {
   2:      HandlerResult ShipOrder( string orderNumber );
   3:  }

I expect to call something like this:
   1:  var result = eventManager.For<IOrderEventHandler>()
   2:                           .In( currentContext )
   3:                           .TryExecute( x => x.ShipOrder( orderNumber ), exec => { 
   4:                              currentOrder.ShipOrder( orderNumber );
   5:                            });

This will execute the given code if, and only if, the HandlerResult of the ShipOrder event handler returns HandlerResult.Success. The variable, result, will hold the HandlerResult so I can take action based on the result (throw an exception, rollback a transaction, etc.)

Now, from the UI perspective, we can have a special UI event handler:
   1:  public interface IOrderUIEventHandler {
   2:      HandlerResult DisplayDelete();
   3:  }

In the backing model, I can lazy load the permissions:
   1:  public CanDisplayDelete { 
   2:     get {
   3:        if( _displayDelete == null ) {
   4:           _displayDelete = false;
   5:           var result = eventManager.For<IOrderUIEventHandler>()
   6:                                    .In( currentContext )
   7:                                    .Validate( x => x.DisplayDelete() );
   8:           
   9:           if( result == HandlerResult.Success ) {
  10:              _displayDelete = true;
  11:           }
  12:        }
  13:        return _displayDelete;
  14:     }
  15:  }

The key, of course, is for the event manager to be extremely fast. I have tuned the hell out of it from a library persepective, but until I get it in a real application, I don’t have much to go on, but I expect the performance to be negligible. The only area where I need to be cautious is in the design of the event handlers themselves.

I also expect to fine-tune the syntax as it's a little "wordy" for me. I expect that I should be able to optimize the syntax down to a single-line evaluation:
   1:  eventManager.Validate( x => x.DisplayDelete() );

As I said, once I start using it, I'll be more apt to tune it.

EDIT: To be clear, this design is meant to address a particular problem: there may be a lot of entities interested in particular events and the decisions made may require input from multiple sources. This is not meant to delegate all functionality or business logic to event handlers.

Brookstone, Stop Spamming Me

by Ryan 16. December 2009 11:19

I'm sorry, this is spam, plain and simple.  I don't even know how I got on their mailing list; I've never shopped at Brookstone.  Note to Brookstone: one email per day MAX or you'll lose a lot of potential customers, like myself.  I took the liberty of reporting them all as spam in hopes that Google will automatically tag them as such in the future.  Thank you, and good day.

Current Progress and The Epiphany

by Ryan 3. December 2009 11:09

It's been about two and a half weeks since my last post.  My project is coming along nicely; I've written a total of about 100 lines of code.  I've spent at least an hour, every day, working on some aspect of the site.  I guess my point is to illustrate how much planning needs to occur before you even write a single line of code.

I think of every application as the creation of building blocks for my next application.  Given this, I tend to labor over the details to ensure that I'm creating a solid framework and not just a one-off web site. While I'd agree with anyone that labeled me as neurotic or obsessive-compulsive, I think this kind of mentality bodes well for a programmer.  

I see programming as a craft, and, as such, I'm always looking for ways to better myself.  In an effort to "future-proof" my design, I'm forced to bring in new technologies and push myself out of my comfort zone (though I'm never really comfortable stagnating.)


One example of this is my epiphany with regards to test-driven development (TDD) (or behavior-driven development.)  See, I find myself stuck in traffic for at least an hour every night.  During this time, my radio is either off or playing a programming-related podcast.  As of late, I've listened to all my podcasts, so it's just me, the road, and my thoughts (and thousands of other motorists).

So, as I'm working through a particular aspect of the framework of my application in my head, it suddenly clicks: this is a perfect place for TDD.  You see, my mind kept wanting to traverse the tree of functionality all the way to the leaf nodes, but right now I only really need to set the stage/build the trunk for those nodes.  I was spending all of my brain power trying to decide how to flesh out the entire design, when I had a solid starting point that could/should be the focus right now.

When following the single-responsibility and open-closed principles TDD is just a perfect fit.  You can create your "trunk" and then mock out all of the nodes.  You create the nodes later and test them independently.  Suddenly, it all made sense.

You see, prior to now, my development workflow was never compatible with the TDD workflow.  I was always trying to make sure I had all my nodes in good order, then put the trunk together (or some amalgamation of that.)  While I've always fully embraced unit testing, I never bought into the test-first pattern.  But, it occurred to me, that it's not "test-first" its more "behavior-first" or "foundation-first."

This epiphany would have never struck me if I hadn't spent the last couple of weeks writing some test code and trying out Moq for the first time.  Once I started to see what these tools offered, I realized the development friction would actually be lessened by viewing the application's development with a TDD bent.  Then, with enough mind-numbing focus during my time in traffic, all of the puzzle pieces fell into place.

I have to say, this new perspective has added a new level of excitement to my development; it's empowering.

With that, I've settled on a tech stack:

  • MS Windows Server 2008 R2
  • MS SqlServer 2008
  • ASP.NET MVC 2 (beta for now)
  • nHibernate
  • Fluent nHibernate
  • Moq
  • MS Testing Framework (I like the integration with the IDE)
Now, my choice to use nHibernate is the topic of another discussion.  I have always been firmly against OR/Ms, but nHibernate impressed me so much that I decided I couldn't go wrong - as long as I take to the time to understand how nHibernate is expected to be used.  

I will say that I started down the LINQ path with great enthusiasm, but it really had all of the bad elements of an ORM and none of the good.  I can't live in a world with out POCOs, sorry.  I will not "work around" any library - they are supposed to help make things easier, not bleed their abstraction all over your code. *stepping off my soapbox*

Perhaps I'll go into my decision to use nHibernate after I get some hours behind the wheel so I can reflect on my decision.

 

 

Finding Balance

by Ryan 17. November 2009 13:09

I am moving forward with a side-project that I hope some demographic will find useful.  I think it's important to note that I'm not motivated by money; I'm not trying to turn any kind of profit.  This project is just for my own satisfaction and to keep my skills sharp and current.

I have to imagine that a lot of startups fail because they focus on the "becoming a millionaire/getting bought" goal instead of the, more important, goal of providing a useful user experience.  I think the greatest sense of accomplishment comes from people choosing to use your application.  It's a big "thanks, we needed this" kind of pat on the back.

I'd argue that even if nobody uses it, it's still worth building.  I expect to learn a lot about transforming an idea on paper into a fully-functional website that's fit for public consumption.  I've had plenty of experience with this in the past, but with other people's ideas, never my own.

As it stands, I'm about 3 weeks into the project and I haven't written a line of code.  Thus far, it's been about planning, use cases, wireframes, domain-name choice, etc.  These are the types of things you need to be willing to devote yourself to if you plan on being successful (not that I have a success yet.)  I find a lot of people want to either hit the ground running and throw together a prototype that evolves into something that people want to use, that inevitably needs to be rewritten at the cost of a great deal of pain in data migration and other assorted idiosyncrasies.  Then there's the other type who want to have a vast landscape of foundation code perfectly architected before producing anything that any normal human being could appreciate.  The key, as always, is finding a balance.

Outside of the balance of design and time-to-market, is the balance of friends, family, work and free time.  It's really had to stay committed to a side-project when you find yourself trying to keep all of the spinning plates from crashing to the ground.  Having a pregnant wife and a 4 year old daughter make this even more challenging.

Again, it's about finding balance and making sure that you're happy doing whatever it is you decided to to do.  If you're not happy to be working on a side-project, then that's called work, and you already do that 9-5.  To be successful, it really needs to be a hobby that others just happen to be able to enjoy.  And, if you make money along the way, good for you.  You can always exceed the expectation that nothing will come from your work, the same can't be said about wanting to become an "instant" millionaire.

 

 

Asking Too Much

by Ryan 3. November 2009 13:43

You really shouldn't ask this of a user of a website.  First, I don't expect to have to wait 5-7 minutes. Second, you're asking me to careful not to close your tab which is pretty much unusable.  Why not just say, "We'll email you when your account has been provisioned"?

My Toolbox

by Ryan 28. October 2009 19:51

I wanted to create a list of the free (mostly) utilities that I use on a regular basis.  I've found that a lot of people don't seek out tools to make their lives easier.. strange.  This is not an exhaustive list, but it's a start.

Link Shell Extension
Create hard and symbolic links in windows
http://schinagl.priv.at/nt/hardlinkshellext/hardlinkshellext.html

Notepad++
Great general-purpose text editor
http://notepad-plus.sourceforge.net/


FastStone Capture *
One of the best screen capture apps out there. Simple and effective.
http://www.faststone.org

7zip
Best compression tool available
http://www.7-zip.org/download.html

Ditto
Great tool for keeping a clipboard history. Great for refactoring.
http://ditto-cp.sourceforge.net/

Virtual Clone Drive
Nice free ISO mounter - works in Windows 7
http://www.slysoft.com/en/virtual-clonedrive.html

XPath Visualizer
Nice tool for testing XPath expressions without the bloat of larger multiple-purpose XML tools
http://xpathvisualizer.codeplex.com/

XML Explorer
Nice light-weight tool for working with XML
http://xmlexplorer.codeplex.com/

SyncBackSE *
Great tool for filesystem synchronization, there is a free version
http://www.2brightsparks.com/downloads.html

GrepWin
Coming from a Unix world, I need my grep. Search and replace, fast.
http://tools.tortoisesvn.net/grepWin

Mozy Home
The only online backup service I trust. Currently hosting 300GB of my precious data.
http://mozy.com/

TortoiseSVN
The only SVN client for Windows that's worth mentioning.
http://tortoisesvn.net/

VisualSVN Server
Great SVN server that run as a Windows service. Takes the headache out of manual setup.
http://www.visualsvn.com/

Snippet Compiler
Great tool for testing expressions outside of an IDE
http://www.sliver.com/dotnet/SnippetCompiler/

Development Specific

LINQPad
Awesome LINQ tool and snippet compiler... awesome.
http://www.linqpad.net/

GhostDoc
Must-have tool for generating meaningful documentation stubs
http://submain.com/products/ghostdoc.aspx

Reflector
.Net assembly analyzer and disassembler
http://www.red-gate.com/products/reflector/

ReSharper *
Must-have tool for refactoring. Version 5 looks stellar.
http://www.jetbrains.com/resharper/

VisualSVN *
Must-have SVN integration with Visual Studio 2005+
http://www.visualsvn.com/

* - Not free but worth the purchase price.

Display Placeholder

by Ryan 21. October 2009 10:55

Whilst playing with MVC 2, I came upon the need to display an element that I could later replace as the result of an AJAX callback.  Starting with the DisplayFor<> HtmlHelper extension, I found that it simply outputs the given value as a string, with no way to later replace it.  I could create a custom display template, but I wanted more fine-grained control.

So, I decided that needed some way to create a display element wrapped in a <span> tag.  With a <span> tag and an id, I would be able to display the model-bound data, but later replace it as the result of a AJAX callback.

This is what I came up with:

   1:  /// <summary>
   2:  /// Creates a placeholder element displaying the value given by <paramref name="display"/>.
   3:  /// </summary>
   4:  /// <typeparam name="TModel">The type of the model.</typeparam>
   5:  /// <typeparam name="TValue">The type of the value.</typeparam>
   6:  /// <param name="html">The HtmlHelper being extended.</param>
   7:  /// <param name="display">The property from the model to display.</param>
   8:  /// <returns>An <see cref="MvcHtmlString"/> containing the display value wrapped in a &lt;span&gt; tag</returns>
   9:  public static MvcHtmlString PlaceholderFor<TModel, TValue>( this HtmlHelper<TModel> html, Expression<Func<TModel, TValue>> display ) {
  10:   
  11:     return PlaceholderFor( html, display, null );
  12:  }
  13:   
  14:  /// <summary>
  15:  /// Creates a placeholder element displaying the value given by <paramref name="display"/>.
  16:  /// </summary>
  17:  /// <typeparam name="TModel">The type of the model.</typeparam>
  18:  /// <typeparam name="TValue">The type of the value.</typeparam>
  19:  /// <param name="html">The HtmlHelper being extended.</param>
  20:  /// <param name="display">The property from the model to display.</param>
  21:  /// <param name="htmlAttributes">The HTML attributes.</param>
  22:  /// <returns>An <see cref="MvcHtmlString"/> containing the display value wrapped in a &lt;span&gt; tag</returns>
  23:  public static MvcHtmlString PlaceholderFor<TModel, TValue>( this HtmlHelper<TModel> html, Expression<Func<TModel, TValue>> display, IDictionary<string, object> htmlAttributes ) {
  24:   
  25:     ModelMetadata metaData = ModelMetadata.FromLambdaExpression( display, html.ViewData );
  26:     TagBuilder tag = new TagBuilder( "span" );
  27:     tag.GenerateId( metaData.PropertyName );
  28:     if( htmlAttributes != null ) {
  29:        tag.MergeAttributes( htmlAttributes );
  30:     }
  31:     tag.SetInnerText( Convert.ToString( metaData.Model, CultureInfo.CurrentCulture ) );
  32:     return MvcHtmlString.Create( tag.ToString( TagRenderMode.Normal ) );
  33:  }

Mixing solid design with code generation

by Ryan 9. October 2009 08:47

One thing about a good software design is there usually exist a number of layers of abstraction.  The problem is that this can start to become cumbersome to manage.  It's important, however, to recognize that there are tools out there to mitigate this and thus your design shouldn't suffer as an attempt to "simplify."

Simplification comes through abstraction and encapsulation.  If you find your architectural framework complex, you're probably not encapsulating the details enough.  I always design with a junior programmer in mind.  That is, could a junior-level programmer pick up my code and be productive in a short period of time.  If the answer is yes, then the design is "easy" to work with, though it may be "complex" under the covers.

The first way to simplify any design and the inherent testing thereof, is to use an Inversion of Control (IoC) framework.  There are many on the market (Spring, Unity, Castle Windsor, etc.)  Such frameworks allow for clear separation of concerns without hard-coding dependencies.  The real benefit is realized when unit testing.  Without an IoC framework, you'd (hopefully) do dependency injection manually, however this would require lots of plumbing to test a single component and lots of manual work.  IoC containers make this easier by "automatically" managing the dependency instantiation.  I great tongue-in-cheek overview can be found here.

Given all of that, I sometime still struggle with the perceived complexity of my code.  Most if the complexity (with respect to dependencies and separation of concerns) happens in the lower layers, from the service layer down.  Fortunately, most of this has a one-to-one mapping to a database.

Let me preface the following by saying that this only applies to websites, as opposed to shippable products.

Now, call me old-fashioned, but I don't like ORMs.  Well, more specifically, I don't like ORMs when there's little chance of a change to the underlying datastore provider.  I don't feel like I get the level of control that I want, and, given my experience perf-tuning applications for production, I feel that I'm justified.  I say this because the ORMs abstract away the database interaction and, following my "junior developer" thought process makes it easy for developers to write horrible data access code.  I feel it's of much more value for a junior developer to become familiar with a particular database vendor (SQL Server in the .NET world, for instance) than an ORM that may or may not be used at the next job they get.  If you understand how the underlying system works, you can then work at a higher level with getter effectiveness.

So, in order to achieve the same effect of an ORM, I simply use code generation.  This is far simpler in .NET than in Java to due to certain language constructs.  By generating the code, I don't waste CPU cycles at runtime as various mappings and pseudo-SQL expression are worked out.  I simply send the query to the db provider.  Sure, this SQL could be vendor-specific, but I find that 90% of the time, I can write ANSI SQL query that SQL Server, Oracle, MySql, DB2, etc could process.  You might say, however, that I am limiting the reusability of my code.  I'd argue that proper separation of concerns shields me from this.  I have never reused data access codes across projects since the data model always differs.

Now, the game changes if I'm building a product.  In this case an ORM is, by far, the best way to support multiple DB platforms.  It's worth noting that even with an ORM, there's still a lot of plumbing and data containers that the ORM maps which should be generated.  Anyone that is hand-writing files that correlate to a database table need are stealing from their customers. 

In the end, this comes down to using the right tool for the job.  I feel that code generation of potentially native SQL is the right tool for a single-db-platform website and, in contrast, ORMs are the right tool for shippable products that can live on any platform.  In both cases, though, code generation should play a role in the development process.

About Me

thumbnail I'm a software developer currently employed by Oracle*.  I work with Java professionally, but my passion is for .NET.  I have (close to) two decades of programming experience and I'm constantly trying to learn new languages, technologies, practices, etc.

 

Disclaimer

* Emerle.net is owned and operated by Ryan Emerle. The views expressed on this blog are his personal opinion and do not necessarily reflect the views of his employer or clients.

The same holds true for comments posted to Emerle.net; they are the comment posters' personal opinion and do not necessarily reflect Ryan Emerle's views or the views of Ryan's employer or clients.