Surfulater got to a point where it was working pretty well and did most things our users had requested and that I wanted of it. There are problems no doubt, such as the poor foreign language support, especially with search. This dates back to a poor implementation decision I made in the very beginning of Surfulater’s design, one which cannot be undone or easily fixed, otherwise it would have been long ago.

To resolve the foreign language support problem and address some other issues requires an extensive and expensive rewrite, which can’t be justified. Furthermore I was beginning to see huge changes in the direction certain types of applications were heading and in the technologies being used to develop them. So it was time to step back and rethink the entire way I develop software, the technologies I use where all this fits in with Surfulater.

The huge game changer in my opinion is the move to applications that run on multiple devices (Desktop PC’s, Tablet’s, Smart Phones) and enable you to access your information on any of these devices, from anywhere. Furthermore you can add and update your information and have the changes available on each device automatically and in real time. This is where I’ve wanted Surfulater to head for quite some time, however it will never happen using the current C++ Windows Desktop code base.

To move forward it was time for a major change, time to move out of my comfort zone of Microsoft Windows Desktop Software development in compiled C++ with all the tools I’ve been using for many, many years to the new world of applications that run on multiple devices, using dynamic languages, with user interfaces that are rendered in HTML, and have severely limited local file systems.  Hard to imagine a more different world, but that’s what I’ve been doing most of this year.

This new world revolves around developing in HTML5, Javascript and CSS and packaging that up into applications that will run on a variety of devices. It has and continues to be a huge learning curve, one that is taking a lot of time and resources.

There was no way I was going to jump into the deep end and try to redevelop Surfulater, as I had way, way to much to learn first. So after much prodding from an old friend I began work on a project with him to create a range of iPad (Tablet) applications. Apart from taking quite a long time the results so far are great, with the first app nearing completion.

In essence I’ve designed and built an application that my partner Stefan will use to generate applications.  So there are two actual applications: the Builder that he uses and the Client Apps that the builder creates and that we’ll sell.

Both of these are written in HTML5, CSS and Javascript. The builder is by far the most complex, using a client-side database, HTML5 templating, local file system access, remote server uploads and downloads etc. It uses jQuery, jQueryUI, Knockout.js and a range of other Javascript plugins and libraries. It is pretty slick and easy to use, enabling Stefan to produce iPad applications quickly.

The end-user client applications use jQuery Mobile and jQuery and are packaged using Phonegap and other tools.

All up I’ve written around 7,000 lines of Javascript so far and have learnt one heck of a lot along the way. And this time around I’ve designed everything to work in any language! There is still some way to go before we’ll be ready to start shipping the first apps, but we are making excellent progress. Stefan was only able to start using the first Alpha release of the Builder about 6 weeks ago and has already completed the first application, which happens to the biggest and most complex of the series of applications we’ll be producing.

On the side I’m am spending as much time as possible researching ways to accomplish my ideal Surfulater type app, with information replicated on all devices. To say this is complicated is an understatement. The research alone is very much one step forward, two steps back, however I’ve never been one to be easily deterred.

So that should give everyone a clearer picture of what is happening in my world at this time.

If I don’t post again before Xmas, and given my track record of late I probably won’t, I wish you and yours a very happy and safe xmas and all the best for 2012.

Neville

PS. It’s nice to have blogged again.

Thank you for your email. Allow me to share my musings on technology.

I will reflect on what you have to say, as I do believe there is some sense in it. I will try to regularly saved pages as they arise; maybe it would not be as cumbersome as it seems to me. I guess maybe what I would want would be to turn it off and on. I will do google searches and click from page to page with a dozen windows open, most of them with multiple tabs and I really do want *every* one of those pages saved (automatically, I think as it is a good record of my train of thought). I guess I thought perhaps there was sophisticated enough searching such that it would not be a waste of time. I also thought it would be often me doing a more general term search, thinking “I know I read something about “dopamine reward pathway” and “the hedonistic calculus” and then I would basically hit on the motherload of all the pages I was looking for. There were more than a few times when I was glad that slogger automatically saved everything. For some reason, though, when I did a general Microsoft system search the results did not come up for terms listed on the html pages, so the search was effectively useless, except for some times when I kind of tracked stuff and was glad it was all there.

1) Slogger is not compatible with the latest version of firefox and I have other add-ons which I believe require it to run. I’m not sure about keeping an earlier version of firefox.

2) I had the issue of a term in an html file saved via slogger not seeming to come up, making search almost useless (again unless I could track it down manually).

3) I guess I just think it is really cool to pick a day from 2005 and just go from page to page that I was visiting and recapture everything I saw. This fits in with my whole vision of the ideal that we will archive experience. As storage space becomes effectively free and camera technology becomes miniature, it will not be uncommon for people to, for example, simply videotape their days. I believe there are people who already do this JustinTV(?). All of this is very crude now. But if the data is gone–one thing we know it will stay gone. Who knows what kind of smart searching technology we will have in 10, 15, 20, 1000, 10000 years. This doesn’t even get into backing up one’s memories from the day so nothing is lost at all. Again, I feel like this the way I originally felt about gmail. Why would I want to save every email I have ever sent? What’s the point? Well, when storage space is effectively free, the question becomes one of search. Clearly search is going to get A LOT !!!! more sophisticated. (Again, though, if the data is gone, it’s gone.) I look back now (being 34 years old) and my parents never had a videocamera, so there is basically no record of my childhood other than a few hundred photos maybe. Contrast that with a kid today who would have tons of videos of all kinds of things. Tons of pictures. And a record that is always there of everything that passed in front of his eyes if it ever occurred to him to want (given that it was effectively free to preserve it). Why *not* do it? You point to security. OK, I guess maybe I should be more concerned about that I am. Maybe someone could access my files from my external hard drive. And, yes, there is the danger of how secure is the data if it is sitting on an external hard drive sitting right next to the computer. Again, maybe not very. But if it costs nothing to do so, why not? OK, search is cumbersome. It seems to me that if there were some combination of smart searching where I could get little thumbnails of the 300 pages where the term came up (I’m taking what would seem to be the excessively cumbersome number of results which supposedly would happen enough to make the whole idea questionable–I don’t think, if the user interface was well done that it wouldn’t be worth it to go through–I think that would could be cagey in how one employed search. I must I admit I was hoping for some exciting things on that front. I guess I want to have all the files there so that, when the smart search technology is there, I can say to it (as it looks over the exabytyes of data I have) in natural language: “Molly” (I’ve just decided I’m going to call my first smart search interface avatar “Molly”–see Kurzweil’s current crude “Ramona”) search my files for the summer of 2009 and display my wall display the first 25 files which use the words “artificial intelligence” and “Moore’s Law” more than twice each in each file (ordering from top left to right and down being the default). And then I walk to the wall size display and scan the different documents, tap the one I want so it shows up on my tablet and sit down to read the document. (or whatever and however it all plays out). Of course this doesn’t even get into more sophisticated artificial intelligence-based smart searching. (I’m sure that in 10 years the above scenario will seem locked into a Windows 95-esque paradigm.) Or, based on a cross-referencing of the relative status of the authors of all articles downloaded in the last 5 years based on (some kind of) Google-like popularity rank, give me the articles on “genetic algorithms” by those writers linked to with the greatest frequency. (Obviously, I’m just making shit up now.)

4) But the key to all this is: We could have Turing-passing intelligent software agents but that won’t do you any good if you haven’t saved the files. And, if the searching is sophisticated enough, you will want all the files you could have.

5) also I saw surfulater as offering files which would go beyond simply firefox. Word. Powerpoint. Adobe Acrobat. etc. All those would show up in the proper order so that when I had 3 word files, 10 firefox windows (8 of which have multiple tabs), an Adobe Acrobat document, and a Powerpoint file; Hey, who knows, maybe a mp3 file too. All of that would be right there so that when I looked at the thumbnails on my wall display I could easily tap any one of them to have it come up in the form I want.

interview with Kurzweil, the man Bill Gates calls “the best person I know at predicting the future of artificial intelligence,” where he mentions backing up our brains. If we can back up our memories (and presumably search them), presumably it would be feasible to search through the text of some html files we went through 25 years previously. Clearly, artificial intelligence programs could see relationships between the data that we can’t imagine. If we are relearning it, there could algorithms for reintroducing the material to us in a way that would allow us to optimally retain it, and it certainly wouldn’t hurt to have a complete record of when we came across it. I realize that technology is hard to make and these are pretty unrealistic, fanciful notions–for now. My hat goes off to you for you will be creating the technology of the future. It’s pretty exciting though I would have to say.

http://www.informationweek.com/news/internet/showArticle.jhtml?articleID=195200003&pgno=1&queryText=&isPrev=

some links: http://news.bbc.co.uk/2/hi/technology/6287126.stm

I apologize if I came across as snippy. I did not mean to.

I guess it just comes back once again to my main point: We could the most insanely advanced technology (e.g. storage and search capability) but it is only as good as the set of data available for search. Pretty axiomatically, assuming your search technology is sophisticated enough the more info you have the better. ONCE IT’S GONE, IT’S GONE. And I want it all. It’s really that simply. If I can’t use it all now, then I want it for later. (I guess I thought I would still be able to search it now to get what I want, but maybe that would not be the case.)

Like I said, I’ll check out surfulater for a while and see if it, in combination with Google automated (link-only) history saving, is my best bet until something more complete comes along. In the meantime, obviously these technology speculations are quite fanciful, and, although I dispute much of what he says, once one reads someone like Kurzweil, one starts to think: OK, what would I want, assuming everything was *exactly* the way I wanted it to be. ANd, then, since it never is, one finds what one can to most closely approximate that desire. Part of maximizing that desire may involve a certain planning for the future. Once it’s gone, it’s gone.

My reply: Thanks for your erudite and lengthy response, which I have to say was rather unexpected. I agree with your view that once its gone it gone, however a very large proportion of what is on the Web tends to hang around. The problem is often finding it again. But you aren’t just talking about the Web I know.

If I look at my personal use of saved knowledge the focus is largely on recently created and gathered information, with anything much beyond a year or two old unlikely to be relevant to me any more. In fact often times it boils down to information garnered in the last few months. Such is the rapidly changing world we live in. However if there is one stand-out thing I’ve learned from developing Surfulater it is that so many people use it in so many different ways for so many different things!

Your vision for search is very good, but we seem to be taking only baby steps in getting there. I have no doubt this is an area that a heck of a lot of resources are being ploughed into though.

Assuming search does improve dramatically and that all Web content is cached and lives forever then I get back to my earlier blog post and question why you need to save each and every page you visit. If you add value with notes and organization (tags/folders) and therefore make it easier to re-find, then that is a different story.

And further correspondence from Mr. T.

Here is a little more I why more unfocused writing on my rationale for getting surfulater in the first palce: I am in the 6th year of my PhD program in philosophy with a strong interest in cognitive science broadly construed (neuroscience, psychology, anthropology, artificial intelligence, etc.), so I spend many hours a day reading dozens of articles (some thoroughly, some only skimming them). As I said, many times during the day I will have a dozen firefox windows open (many of them with multiple tabs) in addition to a couple word documents, maybe an pdf file and a powerpoint for the class I am teaching. This has been my life for the past six years. I would say that just about anything I looked at relevant to my research or just general interest would be something that I would be very interested in being able to find very easily (and I like the idea of recapitulating all the things/files that were going on when I accessed a given, say, journal paper. Of course I would like to take the time to add notes and such, and maybe surfulater will be the key to organizing it, but I can’t tell you how many times I have had the experience of reading something about a topic and being vaguely familiar with it, but not knowing what I read on the topic. I guess I could just get in the habit of right clicking and selecting. (There has to be a key command though, right? I guess I’ll find that out.)

Here is the message I sent to a friend who asked why I would need such a thing when I basically asked if anybody knew of something like “slogger but much, much better” (and to which a friend replied by recommending surfulater):

I guess maybe I am just trying to justify being a packrat but I have something like the following experience quite often. I will be reading one article which will lead me to one web page which will lead me to another page which will lead me to do a google search for one term then I’ll look at one of the articles referenced in one of the articles.

I might also open up Microsoft word and type out some thoughts I have on it. Maybe I get an idea. Maybe there is something I want to check out more.

And then, when it comes to go out, I have to turn off my computer and there is all of this–stuff–that is lost. I feel like I can never reproduce the train of thought I had going at the time. If I had displayed right in front of me chronologically or searchable via keyword every single document that passed in front of me it would jog my memory much easier; it would make it all fresher in my mind when I go back to the same topic or something similar. When I get the feeling, “Haven’t I read something similar on this topic?” unless I have saved it in the right way in my current system by author I can’t quite get what specific article or web page I may have read–forget about having the whole 2 hours of surfing/annotating/reading/switching-between-files laid out in front of me. I guess I feel overwhelmed with information and if I have some logging of the stuff that I have gone through on some level I don’t feel like I am starting from scratch.

I guess I am just a (digital) packrat. I guess I just really like the idea of clicking on a day from years ago and seeing the random books I searched on amazon, the blogs I read that day, the hockey games I checked on ESPN. When storage is effectively free, it is like being able to have a little museum look into the past–your own past. Maybe I should just let things go…

Hopefully, I will find a way to use surfulater such that it will best search my needs. I haven’t begun to tap into all I’m sure it is capable of doing.

I suppose you could use the above, too if you like.

And, I just thought of the work of the philosopher Martin Heidegger. Now, I don’t go into this kind of existentialist philosophy so much any more because it is all just a bit too wooly for me, but one of the ways Heidegger saw us living our lives is by projecting our lives out into the future. We see ourselves as engaged in a project which will define our future. “There’s no future in this.” Or “We don’t have a future.” being ways of saying, “It/You will no longer be a part of my life (or even a part of who I am).” And because we are temporal beings, we are always living out our lives as projecting into the future by appropriating elements/experiences from our past. If you don’t think that’s true then listen to the absolutely horrifying yet fascinating story of Clive Wearing, the person with the worst case of amnesia ever documented, such that every single moment of his life consisted of him remembering absolutely nothing that had ever happened to him and feeling like he was waking up for the first time ever. Interestingly, there were two areas which allowed those moments to stretch more than a few seconds. If you’re interested, I’ll leave it for you to find out what those were. http://www.wnyc.org/shows/radiolab/episodes/2007/06/08 (Part 3)

Anyway, to return to the Heidegger, here’s what I hope is not too boring of a biographical interlude. I studied philosophy in college. I enjoyed it very much but I could never have seen myself become a philosophy professor because I associated philosophy with the kind of wooly thinking exemplified by Heidegger. So I forgot about philosophy and just kind of put it away as something that had conditioned who I was but was not relevant to my life.

A new chapter in my life started: I moved to Japan. So, let’s take the example of learning a language. I happen to speak Japanese. When I lived in Japan, I saw my ability to speak (and read, etc.) Japanese as integral to who I was. I imagined future careers as a translator. I was going out with a Japanese girl. It was very much a part of who I was because I saw it playing a key role in my future. So all of my kanji flash cards and textbooks on Japanese were very important in my life. It is easy to see how I would want to capture all of that material as it went by on my computer.

Fastfoward 7 years later, I basically have no real desire to keep up my Japanese, because I don’t see it playing much of a role in my future, but now, for reasons I won’t go into, I have rediscovered the joys of philosophy. All of a sudden all that old stuff I was studying as an undergraduate is immediately terribly relevant. I am going through my old notebooks. I am trying to get into my head as a student at the time. This is all the more prominent given that I am an adjunct professor, so I am teaching students the very same material I learned. I am remembering back what I did and did not like about philosophy.

What I have meant to indicate by this little exercise is 1) the nostalgia factor which I think is inherently interesting for a packrat like myself; I recently read that one million people have some form of unhealthy hoarding behavior, so I don’t think my wanting to save everything is terribly outlandish compared to some of these people. For me it just takes up space on a $100 TB hard drive, which will probably be $20 next time I buy one.

2) perhaps, more importantly, we may not *know* what it is that we will really wish that we have saved in the future. If you go so far as to see our very selves in the process of reappropriating features of our earlier experiences in ways that project our lives into the future, one can feel a little bit like one has lost a part of one’s self when all of what one did all day is lost (or maybe when only the “current stuff I need for work” is saved). This feeling seems particulately acute for someone like me who, lamentable as it may be in some respects, spends, let’s say 8+ hours a day on the computer.

[The careful reader may notice that, despite my saying how I have put behind me this whole Heideggerian way of thinking, it has obviously had more of an effect on me than I may realize, for I am reappropriating it here, even though I disavowed its importance just a few lines earlier! Surfulater-related, I would love now to be able to look at a complete history of everything that I did on a computer for an hour as a senior in college in 1996. I think such an experience would be a fascinating window into my thinking at the time, and this would be something interesting (to me, anyway–maybe not many others) even without any kind of advanced search capability. This gets part of what I mean when I am trying to get across how, although I would very much like surfulater to organize my experience of using a computer and retrieving files for my research, I would also like to have it as a window into my experience (which, given current technology, there is no hope of recapturing–here I’m thinking of something along the lines of the movie Brainstorm where there is a cap that you can put on your head and experience whatever someone was experiencing at the time a recording using the cap was made).

I guess that’s all. I imagine this voices everything I have to say on these matters. Thank you for stimulating me to reflect on these issues. I think I may even bring up one or two of these points in my “Introduction to Philosophy” class when I discuss Existentialism. One is always trying to make these old-timey philosophies *relevant* to teenagers technology-obsessed lives.

Digital Packrat with an Eye to the Future

Clearly retaining and refinding information is of interest to us all. The amount though is very much an individual thing. Mr. T’s musing’s are definitely food for thought as Surfulater continues to evolve.

And now to the real reason for this article. I’ve always been very interested in getting hold of real life user stories and I know our customers are interested in reading about how others are using Surfulater. J.William LaValley MD kindly stepped up and offered to write an article on his experience with Surfulater, which I present here in full and unedited, of course.

Why I use Surfulater.
 
I’m a “biogeek” physician who uses the internet for many hours each week for medical science research.  My projects require the ability to accumulate large amounts of related scientific articles and the capacity to access them, with annotations, comments and related links – quickly, efficiently and reliably.
 
I must be able to find new undiscovered relationships among complex textual and graphic information that has not been described before.  In the course of this study on the internet over the last 9 years I have tried numerous different applications to help me capture, store and organize the massive amount of information in this endeavor.
 
For a year I used Onfolio and it was slow, very cumbersome and inefficient – and it frequently crashed.  The files created in Onfolio frequently corrupted and could not be re-accessed – “not good”. 
 
Next, I tried Mind Manager.  It helped me map out the general organizational structure of my projects and to link data ‘notes’ and internet links to various topics.  Mind Manager was a better solution for me than Onfolio.
 
However, when using large amounts of textual data and related files (and I do mean really large) Mind Manager was (and is) slow and laborious to capture, link, organize, re-access, and use the information.  The biggest problem is the Mind Manager would frequently “freeze” when I tried to link the Mind Manager topics to Microsoft Word and Microsoft Excel files.  Sure, Mind Manager can perform this function as it describes…the problem is that it takes a l-o-n-g time to do so when there are multiple topics linked to multiple portions of the same Excel file – and frequently the entire app just “freezes” – not good.
 
The result is the data is lost in Mind Manager and the app had to be closed down, restarted and the work was often lost – “worse than not good”.
 
Then, I stumbled onto Surfulater.  “Surfulater” seemed like an odd word to me – yet it made sense because internet surfing is such a big part of my work.  Surfulater has trial version that is risk-free so I tried it.
 
Wow.  My professional life changed.  Surfulater is literally saving lives by the amazing functional ability to gather large amounts of specific, targeted internet information and data – “on the fly” with just a few clicks.
 
Surfulater allows me to now literally ‘zoom’ through large amounts of information very quickly.  With Surfulater, I immediately and easily organize information, link new data to related data in the same file, link new data to related data in external files and folders on the same and other hard drives, and efficiently copy data to easily accessed related databases.   Surfulater lets me easily include comments, highlighting, do text formatting and editing, capture graphics, add links from related web pages…and much more.
 
Quickly and easily, from within Surfulater (and without opening my email application) I send the captured data to colleagues by built-in email function that automatically loads their addresses with a single click.  They can view it in their email without Surfulater.
 
I send this same data by email to other Surfulater users who plug it into their Surfulater databases and can now use it for their work.  I use Surfulater to create quick and simple web pages of simple HTML from the data that I have created myself.  Amazing.
 
Surfulater lets me surf swiftly while nearly effortlessly ‘scooping’ up important relevant information, into efficiently organized, easily accessed information.
 
Surfulater organizes my information in simple-to-use tree format.  Links made in Surfulater are lightning fast – there are no “freeze-up” delays in Surfulater when you are surfing fast, capturing information quickly, and linking it for re-access later
 
Surfulater lets me search any information in the Surfulater database by text word and returns each ‘hit’ with a highlighted reference.  I can see each “article” with one click.   Surfulater lets me sort the information by date captured or alphanumeric order of the title of the “article”.   Surfulater has advanced sorting features that allows me to sort sub-topics only without having to sort the entire database.
 
Surfulater lets me copy and paste sub-topics of one database into other Surfulater databases in a simple 2-click step – “very handy”.
 
Surfulater is the best data-gathering tool for any serious internet surfer.  The Surfulater Forum is actually relevant and helpful for answering questions, solving problems, and requesting new features.
 
Surfulater creator and code-author, Neville Franks, is extraordinarily responsive in the Surfulater forum and in developing customer-requested features in each update.
 
If you surf the internet and you want to capture, organize, save, inter-connect, link, search, re-access, send, sort, and otherwise use the information for any reason, then your best solution is to “surfulate” with Surfulater.
 
A dedicated Surfulater

 J.William LaValley MD 

Thanks again William. If you would like to follow in Williams footsteps we would love to hear from you. Contact details in the usual place, here. 

Synchronization enables you to use the same Knowledge Bases on say your Work and Home PC and have them automatically kept in sync, so you don’t have to manually copy them back and forth. I use Surfulater on a Desktop and Notebook PC and regularly switch between the two, but first I have to copy all of my Knowledge Base files across, some of which are quite large. Then when I’m finished I have to ensure I copy them back. KB synchronization will do all of this for me, without me lifting a finger. The good news is I’ve got a proof of concept implementation working.

So you must be wondering what’s with the Not Happy! Well during my research I came across a particularly interesting article which I only had time to glance at, and put it aside to read in full later on. Well later on arrived last night and for the life of me I can’t find the slightest hint that the aforesaid article ever existed. I’ve searched my Surfulater Knowledge Bases, looked at the last few weeks articles in the Chronological History, searched my Web Browser Favorites and History on my Desktop and Notebook PC’s and I’ve come up completely empty. In complete exasperation I used Google to search for the terms that I thought should locate the Web page for me, worked through pages and pages of results, tried other search terms and after 2 hours gave up.

There is a lesson to be learnt here, and I for one should know it better than anyone. That’s why I’m not happy!

When I designed Surfulater I went down the path of one big file, thinking this would make Surfulater easier to use, because you didn’t need to go around opening, closing and creating files. It didn’t take long at all for our users to tell us they really did want to use multiple files and Surfulater was changed to suit. Their is no right or wrong way to handle this, although some may disagree. It is a matter of what works best for each individual.

The argument for one big file stems from the point of view that all content should be stored together in one place, and the software provides sufficient means to view and work with various subsets of information, to overcome the problem of managing and working with what otherwise might be overwhelming. For example you could hide all folders except those in a specific branch, or only show articles that match some search criteria. You can think of these as filters which strain out most content, leaving behind only the juicy relevant bits.

Surfulater already has some capabilities that let you restrict the content shown in the Knowledge Tree. For example you can hide all articles except those in a specific folder or hide all articles period, or show all folders fully expanded, without any articles. And there is the Chronological tree view that shows content according to when it was added. These are a good start but we need to do more and will.

Fortunately the tree control I’ve written for Surfulater is very fast and was designed to enable tree items to be shown or hidden at will, without having to re-populate the tree from scratch. You can see this for yourself by using F9 or Show Articles and notice that the tree is instantly updated, even when it contains a large number of items. Without this capability, filtering a large tree would be impractical.

So the groundwork has been laid to enable us to provide more ways to filter information to help you focus on what’s important at a particular point in time. I’ve got several ideas for filters including letting you create your own via say a search string. I’d welcome your suggestions on this.

I need to wrap up as I’m told some of my posts are getting a bit long. I’ll end with a few comments. Multiple knowledge bases currently work best for me. This may change as more sophisticated capabilities are added like filters and keywords, but I somehow doubt it. Trees are well trees, and the bigger they get the more time you waste working the tree, instead of getting things done.

Personal knowledge management (PKM) involves a range of relatively simple and inexpensive techniques and tools that anyone can use to acquire, create and share knowledge, extend personal networks and collaborate with colleagues without having to rely on the technical or financial resources of the employer. Implemented from the bottom up by one knowledge worker at a time, these techniques can increase productivity and enthusiasm and help to build momentum that can overcome the technological and social barriers to top-down, enterprise-wide KM initiatives.

….

Information overload is a fact, not a theory, and there is evidence that most people lack the skills or tools to keep up in the Knowledge Age.

Steve talks about Personal intellectual capital and how employees can increase their value both within an organization and in a broader sense by using PKM techniques.

Getting a grip on the shifting mass of information is an important tactic, but using PKM techniques and tools, individuals can go farther, to enhance their abilities and career potential. Effectively managed personal knowledge assets become the currency of personal intellectual capital.

Surfulater is being used by a diverse group of people to collect and manage all sorts of information and of course build and retain knowledge. This article should be of interest to all Surfulater users, especially those using it within an organization.

There are many ways to organize data: labels, lists, categories, taxonomies, ontologies. Of these, ontology — assertions about essence and relations among a group of items — seems to be the highest-order method of organization. Indeed, the predicted value of the Semantic Web assumes that ontological successes such as the Library of Congress’s classification scheme are easily replicable.

Those successes are not easily replicable. Ontology, far from being an ideal high-order tool, is a 300-year-old hack, now nearing the end of its useful life. The problem ontology solves is not how to organize ideas but how to organize things — the Library of Congress’s classification scheme exists not because concepts require consistent hierarchical placement, but because books do.

Clay has some interesting insights about Social bookmarking systems such as del.icio.us and the problems faced where people classify the same information in very different ways. All in all a very good listen with lots of interesting content. I’ll look forward to hearing more from Clay in the future.

The second article was about the large numbers of qualified IT staff that have been let go from Australian companies in the last 3 or so years in the name of downsizing. It seems that these companies are now realizing that they’ve lost a vast amount of knowledge from this process, knowledge that will be quite costly to recover, assuming of course that it can be. This also ties in with offshore development (and on cruise ships) and makes me wonder whether all of the valuable information that is built up off-shore can be transferred back to its owners, or do they write that off in exchange for the money they save using off-shore development.

As far as I’m concerned building and retaining knowledge and the intellectual property that flows from that is fundamental to the long term success of any business. It’s what gives us the edge.

Mind Mapping tools can also be used for KM, but how practical these are I don’t know.

There are several basic problems with KM software which are difficult to resolve. First is the onus placed on the user each and every time new content is added, to somehow categorize the new information. This may be deciding which tree folder it goes in, or what categories or keywords are appropriate for the content or both. This is an obstacle to our productivity, but one that is necessary if we want to readily locate our information in the future.

Our program Surfulater enables users to defer the categorization decision to a later, hopefully more convenient time, so they can add content without anything getting in their way. Deferring the categorization process is just that though, you still need to make time and put the effort in, albeit later on.

One possible technique which can help us out here is automatic classification of text. In essence you read or parse the text trying to pick out specific tidbits of information, which can then used to place it into a pigeon hole.

Another simpler and cruder technique is to apply rules which are used to determine which folder(s) or categorie(s) new content is tied to. A rule can be a simple search string like: if the word engine appears in the text place the information in the ‘car’ folder. Or it can be more complex search terms such as: engine and not boat or truck or tractor. If you’ve ever setup filters in an e-mail program you will be familiar with this technique. These rules can take some time to set up and get working as you desire, but the results are always predictable, unlike other schemes. You can also use rules to enable information to be placed into multiple folders or categories at the same time. We plan to add something along these lines to Surfulater in the future. Time will tell how well it will work in practice, but anything that removes obstacles from our path has to be a plus.

The next difficult area revolves around the ability to easily find information that you know or think you know is in the KM system somewhere. There really isn’t much point saving lots and lots of information if you can’t quickly find stuff again. I’ll leave this discussion for a future post. Stay tuned.

We’ve organized knowledge into trees, from Aristotle to Linnaeus to Dewey. You get a tree by doing the basic thing of lumping and splitting, and then splitting the lumps until you get to a lump that is too unitary or miscellaneous to bear any more splitting. But lumping and splitting has been constrained by physical limitations. For example:

1. A thing has to go in one pile or another. For Aristotle, this was expressed as the Law of Identity (A is A and A is not not-A), a pretty basic rule.

2. The way we lump and split is the same for everyone: If you own a clothing store and separate it into men’s and women’s departments, it’s separated that way for everyone who enters.

3. The lumping and splitting is done by experts.

4. The person who owns the stuff also owns the organization of the stuff. You can’t come into the clothing store and rearrange it the way that suits you.

5. Lumping and splitting results in a neat and clean order. It’s clean-edged.

But now we’re digitizing information, resulting in a third order of order in which we break the rules of real-world order:

1. Things can go in more than one pile – You put your e-store’s hiking boots under shoes, men’s and women’s apparel, outdoor wear, popular items, items on sale, etc.

2. The arrangement can be different for each person.

3. You or your social group are the experts.

4. Users get to control the organization of the stuff.

5. Messiness is a virtue on the Web.

You can see much of this in the rise of tagging: Users create the metadata and anyone can figure out how to sort through it and organize it. It’s out of the hands of the owners of the stuff being classified.

So, what I’m saying is that we’re moving from thinking that the right way to arrange — and understand — things is to figure out the taxonomic tree ahead of time. Instead, make a big pile of leaves, each with lots of metadata, and allow users to add more metadata and to sort and categorize it as they need.

But there are problems with this, especially with regard to tags:

- One word can have many meanings, and one meaning can have many words. As tagging gets more popular, that’ll be a bigger issue.

- If we form social groups based around how we use words, we run the risk of fragmenting ourselves further, this time around semantics.

- Folksonomies can reinforce homogeneity.

I’m enjoying reading David’s blog and it is interesting to try and see where some of the ideas and concepts being presented there tie in with the development of Surfulater.