I was listening away to some ear blistering metal, as I often do, and an ad really jumped out at me.

If you can?t see what is written in the HTC ad to the right, click on the image.  What is displayed is a cross-correlation of several points of analytics data.  Before I jump right in and start explaining each point, think about what is going on with this ad.  This is by no means just some simple ad, there are a number of things going on here.

First data point.  Sprint & HTC, or whoever it is that put this ad together, has retrieved my listening favorites from Pandora.  Just looking at the bands listed shows that to be self evident.  This also seems to be the most obvious piece of data they could have collected about me, since I am logged into Pandora.  This is probably achieved by some web services or other API that Pandora provides advertisers.

The second data point is not immediately noticeable.  I am still at a loss to explain where they retrieved this data point.  What is it?  Concert dates for bands.  Each of the bands listed in the HTC app that is displayed is a coming show.  Matter of fact, it almost seemed like they had shown me my own HTC, except I don't own one.  :)

Now my location data, I am suspecting probably came from Pandora too, but it is the third point regardless.  All together the ad utilizes geo-positional location, my Pandora music preferences, and pulls local concerts from another source (maybe a Pandora listing too?).  This is a perfect use of preferences to display things that are truly relevant to me.

In addition, they may have just helped to sell me on a new phone for my personal line.  I am up for a replacement and anything that runs Google Droid seems cool, but I?ll admit, with the sneak peaks at Windows 7 Mobile that I've seen and the proposed ability to use Silverlight ? I WILL BE switching from the iPhone when that is released.

Again, conversations give me a zillion things to write about.  The recent conversation that has cropped up again is my various viewpoints of the Agile Manifesto.  Not all the processes that came after the manifesto was written, but just the core manifesto itself.  Just for context, here is the manifesto in all the glory.

We are uncovering better ways of developing
software by doing it and helping others do it.
Through this work we have come to value:

Individuals and interactions over processes and tools

Working software over comprehensive documentation

Customer collaboration over contract negotiation

Responding to change over following a plan

That is, while there is value in the items on
the right, we value the items on the left more.

Several of the key signatories at the time went on to write some of the core books that really gave Agile Software Development traction.  If you check out the Agile Manifesto Site and do a search for any of those people, you will find a treasure trove of software development information.

My 2 Cents

First off, I agree with a few people out there.  Agile is not Scrum for instance.  Do NOT get these things confused when checking out Agile, or pushing forward with Scrum.  As David Starr points out in his blog entry,

"About 35 minutes into this discussion, I realized I hadn't heard a question or comment that wasn't related to Scrum. I asked the room, How many people are on an agile team that is NOT using Scrum?

5 hands. Seriously, out of about 150 people of so. 5 hands."

So know, as this is one of my biggest pet peves these days, that Scrum is not Agile.  Another quote David writes,

"I assure you, dear reader, 2 week time boxes does not an agile team make."

This is the exact problem.  Take a look at the actual manifesto above.  First ideal, "Individuals and interactions over processes and tools".  There are a couple of meanings in this ideal, just as there are in the other written ideals.  But this one has a lot of contention with a set practice such as Scrum.  There are other formulas, namely XP (eXtreme) and Kanban are two that come to mind often.  But none of these are Agile, but instead a process based on the ideals of Agile.

Some of you may be thinking, "that's the same thing".  Well, no, it is not.  This type of differentiation is vitally important.  Agile is a set of ideals.  Processes are nice, but they can change, they may work for some and not others.  The Agile Manifesto covers the ideals behind what is intended, that intention being to learn and find new ways to build better software.

Ideals, not processes.  Definition versus implementation.  Class versus object.  The ideals are of utmost importance, the processes are secondary, the first ideal is what really lays this out for me "Individuals and interactions over processes and tools".  Yes, we need tools but we need the individuals and their interactions more.

For those coming into a development team, I hope you take this to mind.  It is of utmost importance that this differentiation is known and fought for.  The second the process becomes more important than the individuals and interactions, the team will effectively lose the advantages of Agile Ideals.

This is just one of my first thoughts on the topic of Agile.  I will be writing more in the near future about each of the ideals.  I will make a point to outline more of my thoughts, my opinions, and experience with the ideals of Agile and the various processes that are out there.  Maybe, I may stumble upon something new with the help of my readers?  It would be a grand overture to the ideals I hold.

Recently the question came up from a close friend of mine, "will my PhD help me attain a higher income in the north west?"  I had to tell him, that it might get him a little more, but it won't get him in the top income brackets for the occupation.  Another time, a few days later, someone else asked this too.  Then again, I see a job posting that requires a Bachelors Degree and some other nonsense.  The job posting even states they want "A-Game" talent.

I am almost shocked at how poorly part of this industry doesn't realize how unimportant a degree is to getting real top tier, a-game talent.  (and yes, I get a little riled up about this matter)

You Can't Make Good Software Developers.  No college out there is going to train someone to be in the top 10%, and absolutely not to be in the top 5% of skill levels.  Colleges can NOT do this.  It is up to the individual, and the individual alone.  If top tier talent seems to come from a college, one should check their premise and look at the motivations the individuals have to go to that school.  There is most likely a reason that top tier talent appears to be made there.  The college however, can only guide or assist, but I repeat that "top tier talent is a very individualistic endeavor".

Some might say, well a group is needed, support is needed, this and that are needed.  True, an individual needs a support system and a college can provide that, but it generally ends there.  The support group helps, provides a sounding wall, and provides correlation to good ideas for the a-game top tier geek.  But again, the endeavor is the individuals desire.

top tier talent is a very individualistic endeavor - Me

Hiring Top Tier, A-Game Talent

There are a few things when trying to hire this level of game player.

1. The first thing is to not require a degree of any sort.  Sure, it looks good, but it won't dictate anything other than the individual was able to go through the regimented steps of college.
2. List the skills and ideas that you would like to find in an individual.  Think of two people meeting for the first time, what do you want to know about the other individual.  Team fit is absolutely fundamental for top tier talent.  That support group that I mentioned above, top tier talent works best with a solid group of players.
3. Keep your technology up to date, moving forward, and don't bore your top talent if you manage to get it.  If the company slows down, they will leave.  The more valuable they find out they are, the lower tolerance they'll have for this.  For managers, directors, and leaders in an organization this is THE challenge for them.
4. Provide opportunities not just for advancement, but ways for them to advance their knowledge such as training, a book budget, or other means.  Even if some software they want to use isn't used ton the project, get it for them (within reason of course ? couple $100 or even a few $1000 for a good software license to MSDN, Tellerik, or other suite of software is ideal).
5. Don't push them to, and don't let them overwork themselves into burnout.  This, as a leader in an organization is easy to do if one finds themselves actually hiring top talent.  Because top talent just provides results and more results.  But they are human, they will break, don't be the cause of that or you'll lose your talent.

For now, that is it from me on this topic, back to the revenue, code, projects, and pushing forward.

Ok, let me get this straight.  HTML 5 is supposed to offer RIA (Rich Internet Application) type abilities to the browser rendered (i.e. no compile, Just In Time-JIT) web.  How JIT & no compile, markup based, mixed platform technologies are supposed to offer a comparable performance and UX/UI experience seems sketchy at best.  Now let me clause with, I am not writing this idea off, just hesitant to believe that there will be comparable performance to frameworks such as JavaFX, Silverlight, or Flash.  With out of browser Silverlight, or Adobe AIR Apps, or heaven forbid WPF based RIAs there really is NOT a performance comparison.  I can safely assume that Silverlight & WPF (mainly because I've seen them perform) will smoke HTML 5 for advanced rendering or solid advanced RIA style interfaces.  Especially in a Line of Business (LOB) type application.  There just is NOT a comparison in this sense.

The video & audio elements are an entirely different rant.  Sure, it looks great, the O3D demo looks great, but this is still not going to compare to Silverlight or Flash.  These tools are going to smoke HTML 5.  The complexity of getting video, audio, and these other advanced elements of HTML 5 supported has the same issue as Silverlight has for penetration.  Sure, the browsers will start having these things built in, but that will be completed about the same time as Silverlight has similar market penetration as Flash (i.e. the 98th percentile).

Now some, such as Matthew David, wax somewhat poetically about HTML 5.  His article Inside HTML5:  The Browser becomes a first class RIA citizen is an interesting read.  He obviously has great enthusiasm behind this idea.  But I'm not buying it.  The technology is not comparable, HTML is band aided already beyond belief (remember, it was supposed to render documents for linking kind of like a library - NOT for all the advanced things people tend to try and use it for these days).  HTML 5 is merely another band aid, albeit a bigger band aid, then the last several versions of HTML.  The core focus of the markup at this point is basically ignored.

The Positive

The cool thing, I will admit, is HTML 5 does add a lot of options to the browser based web.  I don't think we'll keep going down this rather archaic version of the web forever, but it does provide a stop gap between much more interactive applications.  It also provides a stop gap fix for companies that aren't ready, for whatever reason, to jump on the RIA bandwagon.  HTML 5 will provide great features, but I just don't see it as a prime mover, but more of a stop-gap between the next best way to connect to the web.  Maybe that is Silverlight, Flash, and AIR, or maybe it is some other type of platform or tool.

In the end, something will definitely eclipse HTML 5, but HTML 5 will be great in the meantime.  At least until we run out of fingers to poke in the holes of the dam (i.e. HTML 5).

Outlook and the team responsible for said product, are really pushing hard for the 2010 release.  Of course, I can't talk about it, but yes, I have seen it and it looks & plays awesome!  However, social media got a slight extra kick from Microsoft recently with the Outlook Social Connect.  The Webtrends blog has an entry up Outlook's Social Connect:  Social isn't just AIR toys anymore! by Michelle Anderson.

This got me thinking about various efforts that I know are going on to bridge a lot of this social media deluge.  I get the feeling there are going to be some truly breathtaking applications hit the scene in the later part of the year or even early 2011.  My curiosity hangs around what will they bridge?  Outlook is a prime example of bridging data points that become truly useful information.  With all the addins that Outlook encompasses, one truly becomes a powerhouse power user with a tool like Outlook.

I have been working on some things myself, including proposing a new open source library to bring together services into a centralized library for ease of use.  Anyone got any ideas on that?  Interested?  Thoughts on which services should be brought together first?  The following are the services I am thinking about providing libraries for initially.

• Flickr
• Twitter
• Webtrends
• Smugmug

The reasoning behind these of course, is purely selfish.  I want to have easy access via an API to these services because I use them.  In addition, I figured there are others out there that may like a centralized library.  So toss me some ideas of other services, if anything, I can maybe create a starting point.

Read Part 1 & Part 2 of this series if you haven't already. 

Last entry I worked up the basic origins and destinations for the data we have. Now we have to start turning this into something real, something concrete. The following processes include a whole soup of acronyms and other cryptic vocabulary. The most common thing I will probably use is the ETL acronym, which stands for Extract, Transform, and Load. This is the process of bridging the data to the various originations and destination and moving that data appropriately.

In the last entry I finalized the originations and destinations as shown below;

In my previous entry you may have noticed that I had posted SSIS with the Destination list.  Being one that corrects themselves when mistaken, I took it out, as it does not belong there.  The SSIS is our tool that will perform the ETL functionality for this project.

At this point we are finally going to get into the dirty bits of these pieces of technology, and how we need to tie them together.  I am going to attack them over the next few entries based on the order in the lists above.  The first item, is the Excel & Access 2007 customer relations listings from sales.  Here?s a description and a few shots of what this thing looks like.

The access database is setup with a very simple relational data schema.  This is shown below (click image for larger view).

You can see there are pretty standard pieces of data, in a generally normalized (3rd form for the most part) structure.  This is fine.

Next is a shot of the data entry screen for adding opportunities.  There are respective screens for customers and employees.  Everything needed for a basic customer list & tracking basic things.  Nothing too extravagant here either.  Again, all is fine.

Below is a simple report that shows the available opportunities that are open.

Another report showing the forecasts.

Below is another forecast sliced grid.

So all that seems normal enough.  But the processes are what makes things tricky.  If everyone just managed sales from the database, all would be right in the world.  The first thing that breaks this is that each sales person enters their sales during the day and other information in a spreadsheet that is not linked to the underlying database.  Someone each morning puts the previous days sales information.  This of course, breaks down data integrity.  Below is a sample sheet that is used each day.

One thing that Excel is used for, that doesn?t break reports is the lists of prospective customers to call.  As shown below.

Now that we have a break down of the Excel & Access Customer Relations Management Software, I will move onto the other pieces of technology in the next entry.  This is the data point that has the most prospective data risk, so I put it at the top of the list to cover first.  After I cover each of the systems, we will move into the architecture of the system overall.  So keep reading, more juicy bits to come.

Alright.  Here I am working through some analytics trending of my Twitter, Facebook, www.adronbhall.com, LinkedIn, and other various sources of traffic.  Here's the frustrating part.  I don't mean a little frustrating, I am talking about a-grade I almost want to break something frustrating.  I have bit.ly, Webtrends Analytics, a database with cross-correlated results, a Twitter monitor in another window, and other little apps to track what is going on.  Where does most of this data come together?  In my head.  Why do I have to manage everything this way, it is damn frustrating!

So here is an idea, and I am just thinking outside the box here (as I always do since nobody will let me in the box anyway :p ).  How about these things get merged into a good solid single interface.  Maybe even some of the environments get rolled together in addition to the tracking analytics and reports.  How about a solid dashboard that allows drilldown into the various mediums to allow actual interaction with the data and platforms?  Imagine a dashboard, that reports the analytics for all these things but then lets you just kick in and start tweeting or posting the latest bits to Facebook.  Why have a half dozen or more applications and web sites open.

What we need here is some technology leadership.

Others have discussed this recently and are looking for some results.  Ok, some of you readers may be thinking, "do you not work for Webtrends Adron?  Go build this stuff!"  Well maybe others and I am, but I am still frustrated at not having it right now.  It seems this is something that is slowly slipping by in this industry and we need a centralizing effort for all these data points to help in actually making sense of them.

A fellow analytics & marketing professional, Rodica Buzescu writes in her blog Morphing Through Time a post titled Web Analytics ? Leaving a Lot to Be Desired.  She lays out the current analytics situation very clearly.  There is no tool that really bridges the data together easily.  Some BI Pros out there may say "oh but I have all that stuff together".  This may be true, but how much time and effort did that take?  What we need is a centralized management solution for this data.  Something to bridge the data together to make it actual, usable, and actionable information.

I have also had a number of conversations with Eric Peterson about various analytics on various platforms.  One such paltform is Twitalyzer (@twitalyzer), that he and @katzpdx have put together.  This is a prime example however, of a very useful tool, but something that lies outside of other tools.  The question is, how to integrate and what to integrate.

As an analytics advocate, practitioner, and developer myself I have one huge issue with a lot of my analytics that I use.  Webtrends Analytics & Data Mart help me bring most of those together.  However there are a number of things that are still disparate and segmented in the wrong areas.

So with this post as it stands, what do people want joined?  Where are all these data points that people want?  I think something good just may come out of this desire, but the ideas do have to come together solidly.  I have things I can?t disclose, but I know there are minds here at Webtrends thinking about bridging this gap right here and right now.

Where is the business intelligence?

This blog entry may instigate just a bit.  It will also be a little long for a blog entry. You've been warned.  I suppose though, if you know me & the work I do, that is not really something new.  I see something wrong, broken, or otherwise and I am likely to point it out and describe it in detail.

As I roll into 2010 coding, implementing, and rocking with Webtrends, I have noticed something lacking in the analytics industry.  I will add the clause that obviously Webtrends has people thinking about these things and actively working on this topic, but what I want to point out is a general issue.  Where is the other data, where is the existing data?

It seems, even though some company's kind of get to a certain point in connecting data points, not many really do.  The biggest reason is that most companies are just a few steps away from actually being able to do so.  The other even larger reason is, many do not realize what data should or should not be connected.

When someone starts pulling CRM (Customer Relationship Manager/Management), Analytics, POS (Point of Sale), ERP (Enterprise Resource Planning) data, and other sources into a single reporting repository we finally have real business intelligence.  Otherwise so many entities stumble through the land mines of data confusion.  I see this so much it really drives me crazy sometimes.

So how can a company or entity identify and connect these points of data?  It often starts with a ridiculously simple step.  At risk of oversimplifying things, let me just state the first step in getting out of the data confusion land mines is to first figure out your data.  Ask these things:

• What data does the business have?
• What data is currently used and available?

Do NOT ask what data you want, do NOT ask what may not be.  What you want to know first, and so many companies make this mistake, is to know what you know.  Do not, at the early stage of business intelligence information gathering start asking too many hypotheticals.  I promise the risk of failure increases exponentially for every hypothetical data point added.

Once you have identified what data is available, start figuring out how the data is related.  Once you understand the data you can then, and only then, make the huge leap to determining what data you want and how to get it to where you want.

Let me draw this out in a real world example.  Beware; I am using my creative mind now!

What we have so far, for Awe Widgets Incorporated, is several data points.

• Point of Sale/POS Systems in 300+ stores.
• Web Analytics (by Webtrends of course) tracking all sorts of great data points on the Awe Widgets Incorporated Website.
• Internal Accounting Software (Almost ERP, not really)
• In-house Built Customer Lists for Sales.

So there we go, four key pieces of tracking.  So how would they work together?  With a little further analysis (my key creative side now analyzes Awe Widgets Incorporated internal structure) and we find a few connections.

Correlation, POS to Webtrends Analytics

The POS System has a tracking identifier for customers which we can use to sync up with logged in users tracked via Webtrends Analytics.  This data can be used to derive who is and is not in stores purchasing.  In addition trending could follow the user flow to derive some actionable decisions on how to encourage online or store front shopping.  Just these two data points being connected add a lot of value.

Correlation, Internal Account Software ties to POS

Another data point tie in with the aforementioned POS & Webtrends data is the Internal Accounting Software (IAS).  The IAS holds information related to each sale, and other correlated information about how sales are going for the quarter, year, and other performance indicators.

Correlation, In-house Customer Lists for Sales

The sales department, in aggressive technical fashion has built a number of customer lists in Excel & Access.  The Access Application has a partially updated data store with a server based Excel file holding the updated piece of data about each of the sales person's current sales.  I know, I hear it now, every developer that is familiar with this scenario screaming, "OMG, you have your data in Excel AND Access, and it is supposed to have integrity, and be aaaaaaaaaaaaaaaaggggggggggggggggghhhhhhhhhhh noooooo!"  But you know, this @#$% happens.  : )  When things are like this, solutions get creative.

Tying Together the Pieces

Alright, this is when the awesome nerd bits start to happen.  But I have covered enough for this entry.  In the following entries on this topic I will step through this first data finding mission and start discussions on how to connect these sources and get that data mart, warehouse, or other middle tier piece into action.   I will continue on and lead into how the data can finally start telling a real story.  Because in the end, the real story is, somebody needs actionable data to act upon.  Does it really matter where it is?

Check out Part II of this series. 

Ok, so over the last 30 years or so, Microsoft has made a lot of software.  Probably more software than any single entity on the face of the earth.  The first two decades of software was rough when released in beta.  If it was even released in beta, or alpha, or as service packs.  Most IT people, software developers, or anyone using software would almost always steer clear of beta, alpha, or service updates of software.

Why?

That is the easy thing to answer.  Most of the time, the software would not even work.  At least not in any semblance of the way it was supposed to.

Fast Forward to Today. . .

Software works.  Rarely does one download software that is in beta and it doesn't work.  At least on a fundamental, core functional basis.  This is very much apparent at Microsoft.  The competition lit a fire and Microsoft has answered with more frequent releases, more service packs and updates to keep things secure and up to date, more quality packed into every release.  Why did they make this massive transition?

Agility.

Yup, I said it, the firestorm that has been the Agile movement has caused a major upheaval at Microsoft.  Out went the Waterfalls and in came the Maneuvers of Agile.  In all honesty, if someone is still arguing the tenants of this success, they?re lost in a bygone era of questioning agile processes.  Of course, I will admit, many of the processes are up for debate in various ways.  But the core culture of software development has been dragged kicking and screaming.

Gone are the grognards sitting in the corner being anti-social, hacking away on some code that nobody else will ever be able to maintain.  Gone are those days when a development team works in a vacuum, and fails.  Instead we have faster teams, more customer involvement, iterations or sprints, and in the end higher quality software.

The reason I bring this up really has one simple intention.  Stop freaking out over beta software.  Implement things with it.  Use it.  Love the stuff.  I am regularly using beta software, with all sorts of new goodies, and releasing these projects into production environments.  You might say, "really?  that is so dangerous", but I have not had a single failure, crash, or error related to a beta issue in production for at least 5 years.  Back when Agile was really just starting to gain steam to run the course.  So use new software, stay on the bleeding edge, and help everyone by helping yourself keep up to date!

When creating an Excel Application there are some things to keep in mind.  Recently I was creating some tests, which I'm not sure if they'd be "unit test", but they do test Excel.  I created a fixture setup and a fixture tear down that would create my Excel Application object that I?d want to test again.  The code I ended up with is shown below.

[TestFixture]

public class ConnectionManagement

{

    public Application ExcelApplication { get; set; } 

    [TestFixtureSetUp]

    public void CreateExcelAppAppropriately()

    {

        ExcelApplication = new Application();

        ExcelApplication.Workbooks.Add(Type.Missing);

    }

    [TestFixtureTearDown]

    public void QuitExcelAppAppropriately()

    {

        foreach(Workbook workbook in ExcelApplication.Workbooks)

        {

            workbook.Close(false, false, Type.Missing);

        }

        ExcelApplication.Quit();

    }   

}

Now if anyone has a better idea on how to test Excel and knowing the code will work against the actual Excel Application, PLEASE, let me know as this doesn't feel like the best way to do this.  I keep getting the sinking suspicion that there should be a better way to test Excel Application Addins, Spreadsheet code and such.