personal knowledge management

A friend and I were recently talking about knowledge management. In essence, that strikes to the heart of what I care about: how to organize what we know in a way that is clear and easy to communicate to others. This incorporates writing, philosophy (epistemology), computer science (data structures for representation), etc.

This blog’s real purpose is to start making small strides towards this ideal; figuring out what works and what doesn’t work for myself. And I guess, trying to replicate it broadly. I firmly believe that abstraction or the lens/frame of reference is the key to this, and not enough people notice that. (It’s one of those things that sit at the back of your mind until you think about it, then it colors everything.

So this is what will happen:

Sources from other people:

• Shallow: Hyperlinks / Delicious — bookmarks are key to creating annotalia. People are amazing, I just need to collect their knowledge.
• Deep: Librarything — books are a huge place where I get my knowledge from.

Sources from me:

• Step 1: Textpads, almost no organization. Shorthand. These keep overflowing. Nice to get stuff like project ideas online / into the cloud.
• Step 2: Inditing cycles, brewing ground for unorganized thoughts, translation from shorthand.
• Step 3: Annotalia, final (gamma) of posts. Polished enough for publication. Includes all my source material including my own fill-in-the-gaps.

Note: this division of areas of knowledge + iterative cycles match my own “when-I-have-time” style of writing.

In other words, delicious (lists of bookmarks) + indcyc (personal commentary) = annotalia. Annotalia derives from the concept that ALMOST everything that is worth knowing or being written about has already been written about. The best thing we can do is to help guide readers to it. I’m sure that’s been thought up thousands of times. So my unique take on it is how much thought I can put into any given subject and to expose myself personally, to give people something to connect to.

I think that’s the thing I can make that people will benefit from the most. And writing things in a formal manner for publication helps me get rid of clutter, learn more about material, categorize things in my mind, and have a one-stop shop instead of a giant number of them. Let’s see how it works.

I’ve given this problem a lot of thought throughout the years. If I might restate your problem, it might be: “how do I create a medium for storing knowledge that is optimized for quick and accurate retrieval by myself and others?” To put it bluntly (as a sort of flat joke), you might as well be asking how to write.

Before I go further into why a perfect, or even close-to-perfect solution is impossible, let me record the different solutions I personally use:

When I need to recall something I read somewhere:
* Delicious (index) – keep track of interesting links, tutorials/howtos
* Bookmarks (staging) – keep track of short-term bookmarks for current projects. Move into delicious after projects finish.

When I need to recall influential authors on academic subjects:
* LibraryThing (view) – keep track of what books I’ve read, reading speed, # books read, # books by type read.
* Indite/MindMap (model) – summaries of books, quotes I found memorable.

When I need to recall how I know someone, conversations with that person:
* Highrise (view) – contacts (names, addresses, etc.), favors, conversations.
* Gmail Contacts (model) – contacts and emails, hooks into Highrise.

When I need to recall my mindset at a given time: (I usually choose medium simply through whatever is convienient. I find I tend to put more personal thoughts on paper and more business stuff on computer.)
* Notepads (electronic) – todos, miscellaneous, happy moments, timelines, past projects, current projects.
* Notepads (paper) – I have 20+ notebooks on which I keep a running diary. I sort by month/year to correlate projects with mindframes at that time. (What were my blind spots Senior year of high school when I started that stupid project?)
* Subject leafs (paper) – I keep all my notes from subjects in a giant filefolder for easy lookup. Still use some notes (ie from statistics)
* Classwork (electronic) – Same as above.

Personal data:
* Spreadsheet – keeps password, clothing I like, websites I use (infrequently), health info…
* Handbooks – contain howtos I’ve collected on different tech areas.
* Backups via Dropbox – Music, documents, photos, work, websites, etc…

When I’m managing projects:
* Lighthouse – state of different projects I’m working on.
* Archive (model) – all my archived completed projects on my Linux server.
* Git (model) – backup of all my archived completed projects.
* Portfolio page (view) – archive of completed projects and their start and end date.

Reflections: (Sometimes I link this with mindset at given time.)
* Photologues (view) – blog keeping track of places I’ve seen, pictures.
* Flickr/Facebook (model) – powers the above and other views.
* Indcyc – collection of essays on views on different subjects; things I’ve found surprising.
* Annotalia – collection of surprsing links and essays sorted by different lifetimes and important decisions at every age.

Personal image (somewhat irrelevant):
* Personal page – link to above when I want them publicly sharable.
* Facebook
* Linkedin

Let me try to summarize how I selected the above:
1) Locate need; specifically isolate a TYPE of data that I feel I will need in the future.
2) Create ideal ABSTRACTION/lens around the type of data. What parts of the data do I need to remember? For example, I need to remember very different things about my contacts than my books. One can relate this to creating a database schema. Make a list of database columns sorted by priority.
3) Find service providers that will store STRUCTURED data. Evaluate based on:
a) Security/backup. (SECOND MOST IMPORTANT)
b) Cost.
c) Alignment of abstraction to implementation. (MOST IMPORTANT)
d) Bundled with view?
4) Evaluate the view that is bundled or find a view: (For example, Flickr as my model for images is decoupled from all the views that use it.)
a) Ease of data entry. (MOST IMPORTANT)
b) Ease of data recovery.
c) Speed.
d) Cost.
e) Security/likelihood to go bankrupt.

Let me note right now that your Wiki does not meet the following requirements:
* Alignment of abstraction to implementation.
* Ease of data entry.
The most egregious error is that for any subject, you are limited to entering data in the same way (that is, freetext.) This is also slow. It is also not in a structured format, and thus, cannot be ported to different views.

So I’m leading up to the fundamental problems with having a one-size-fits-all solution:
Everything we have to remember has a class; for any class of objects, we ONLY need to remember specific parts of information. Your needs, in terms of abstractions, will be different from mine. Furthermore, the classes of objects that are important for you will be different from mine. All of this leads to the conclusion that what people are looking for is not completeness but SPEED.

The real question you need to be answering is: what system creates the FASTEST insert and retrieval times for a wide audience. If your system is slower/more cumbersome to use, they’ll simply use something else (like YOU.) The only people who will use your system is the ones for which for EVERY class of object you support, it is the FASTEST. That is an incredibly small subset.

And this is why I expect that people’s time is more productively spent searching/evaluating existing systems than building a new one.

That said, I still think it’s possible to do a better job in several areas, if you’re interested.
- Edwin

>> Excerpt from TODO post:

There are so many different ways of keeping track of tasks:

• Living in text files — emacs also has a todo mode.
• Using a wiki — also a simple 1-file approach.
• Enterprise software — Outlook.
• Project management software — Basecamp, Backpackit, Lighthouse.
• Notepad, organize by cross-section between importance and deadlines. (rpausch)
• Google Calendar, daily tasks.
• Google Mail, todo list OR gtd module.

My personal system has been revised many times and consists of:

• Notepad (todo.ini) — Dashboard view of tasks sorted by priority.
  • The paradox is that one also needs to keep track of due dates. Tasks have specified or vague deadlines if they’re externally motivated or no deadlines if internally motivated.