Obsidian: A Productivity App from the Threat Intelligence Gods

—
I’ve started a weekly newsletter! If you want updates on this blog or my research, or just a weekly dose of the going’s on in information security and threat intelligence, you can subscribe here.
—

I’ve been on a productivity kick recently.

I know, weird start to a blog about threat intelligence, but it’ll all come full circle in a second, just bear with me.

I did all of the typical things that come with a productivity kick, and right now I can just hear the optimistic part of my brain saying “Noooo, this time it will be different!” It might be! I did the trip to the Productivity Mecca in reading Atomic Habits (affiliate link) and Deep Work (affiliate link) and changed quite a few things around to help out with setting and destroying some habits I’ve been meaning to change for a while. Those two books were actually extremely helpful in changing some habits and rewiring my brain to think about work and productivity differently, but that’s probably not the reason you’re reading this blog, so I’ll shut up after telling you to go read those books if you haven’t already.

Part of this “productivity journey” let me find the “second brain” philosophy. Essentially, the idea is that we should use technology for all of the things that technology is good at, and our brains for all of the things that our brains are good at. Computers are great at remembering things, analyzing large volumes of data, visualizing graphs, and repeating automated tasks very quickly. Our brains are really good at analysis of the “intelligence” variety, coming up with ingenious, unique ideas from scratch, writing blogs (well, some of our brains, at least, maybe not my own) and creating analytical works based on data analyzed by computers. The “second brain” is a way of offloading ideas created by the brain, as well as raw data that may be usable in the future, into a place where it can be stored, backed up, and analyzed by all of the features and mechanisms that make computers helpful for humans.

During this period, I came across the PARA Method, detailed in this blog by Tiago Forte, its creator. The idea, put very simply, is to organize your thoughts, files, projects and digital life around Projects, Areas, Resources and Archives. Projects are tasks with due dates: intelligence reports you’re working on, investigations or incidents that you’re looking into, financial goals with specific end times, etc. Areas are defined as running projects, essentially: things that you’re going to constantly be working on long-term, so things like continued development, financial freedom, fitness, areas of study like Math, and work-related subjects that you’ll be studying long term, such as specific threat actors or geopolitical areas. Resources are references that support your Projects and Areas, so individual reports, videos, podcasts, or self-made notes based on what you saw.

Now, all of this sounds like a quirky and troublesome method of tracking your life. It took me a fair week or two to get a lot of my more mission-critical research transferred over to this system, and I’m still moving a lot of my right wing extremist research over. However, the real value of the PARA Method comes when you pair it with a smart note-taking app like Obsidian.

Introducing Obsidian: Linked Smart-Note Taking

Obsidian is a smart-note app. It allows you to record and display notes in Markdown just like many other, similar apps, but its power comes in its backlink feature. I can link disparate notes together into webs of linked ideas, all organized under the PARA Method. So, for example, I have a folder under ‘Areas’ called ‘Russia’. In that folder, I have sub-folders, say ‘GRU’, ‘FSB’, ‘Notable Politicians’, etc. that are all potentially interesting subjects to study over time. When something like the recent treasury sanctions occur that implicate the GRU and a couple of private businesses, I save a copy of that sanction document and link together that copy with the notes that I have on the private businesses, the GRU and the units that were implicated.

Over time, you can open your master-note on the GRU and have a central document on the GRU that is linked to all of the sources and references you’ve used over time. You can then link other entities, such as Fancy Bear, to the GRU and include references that tie the two together linked to both the Fancy Bear note and the GRU note.

This translates into an easily searchable repository of entities (usually tracked in the ‘Areas’ section in the PARA Method) and references (tracked in the ‘Resources’ section in the PARA Method) that link to one or more entities. It also leads to some pretty incredible looking graphs that you can use for link analysis, or just making you feel highly productive as you stare at your beautiful, aesthetic creations.

It lends itself extremely well to automation as well. Markdown, to me, is a great medium between aesthetic readability and parseability: using a simple syntax, you can make readable notes with linked images that can also be parsed and created by automation very easily. I created a simple Python script that parses for IOC’s within a note (think a threat intel blog copy/pasted from a vendor site) and creates individual files per each IOC with associated metadata.

These IOC files are then tied to the individual report, which you can then associate with individual threat actors with further, trivial automation. The IOC extraction script took me… maybe 15 minutes to write, and can benefit from all the power available to Python, such as regex, third-party API integrations, etc. You can literally write a script that extracts IP addresses from a report, automatically look up their WHOIS information, ping the IP using proxies to check if they’re alive, and look them up in Shodan, then dump all of that information into an IOC file or files and associate the individual pivots together and integrate it with Obsidian’s graphing functions incredibly easily.

So, with Obsidian, you can integrate in-depth analysis in IR notes, third-party vendor reports from OSINT or private sources, write automation layers on top of it for automatic or semi-automatic enrichment, and have all of that in one easy-to-use interface, while benefiting from a built-in graphing interface that you can navigate with ease. The organizational methodology is file-system based, so if you don’t like the PARA Method, you can use your own. The interface also has some great hotkeys built in that allow you to make use of templates, so if you want to manually create, say, an entry for a third-party vendor report, you can easily create a template that you can insert with a couple of keystrokes.

Conclusion

My workflow has massively benefited from Obsidian. I can keep my rich text notes with analysis I write myself alongside raw data and third-party vendor and application reports all in one place, with custom automation available with a pretty trivial amount of effort. Obsidian may have been designed for academics in mind, but its power and capabilities lend themselves very well to the average threat intelligence analyst and researcher. I highly recommend checking it out, watching some videos on it and implementing it in your everyday threat intelligence workflow.

By way of disclaimer, I’m not in any way affiliated with Obsidian. I don’t even really know who makes it. I genuinely love the software and have enjoyed the insights and capabilities it gives me as a researcher.

—
I’m not active on Twitter anymore, for reasons I talk about in this blog post, but you can follow me on my Twitter page for new blog posts and announcements.