I'm a programmer. I like to write code and whenever I can, I like to reuse bits of code I've written in the past instead of writing it from scratch.
Lately I've been spending a lot of my time thinking. However, unlike programming, I feel that my writing is repetitive and that I sometimes don't make any progress with my ideas. Oh how I wished writing was more like programming.
There's many ways one can write. You have the top-down approach, where you first lay out a plan of the topics you want to write about. You also have the bottom-up approach, where you write things as they come to mind, sometimes rearranging the content. Generally, when I write, it is to dump what is on my mind about a certain topic, so it is very much like the bottom-up approach. However, the thing I dislike about it is that you can end up with subtle similar sections of content, but not know about it. You actually need to go through all of your writing to discover these bits of duplicate writing. That's a lot similar to writing code and sometimes finding out you have functions/methods that do almost the same thing...
In programming, when you realize that two bits of code are similar, you refactor your code to get one out in order to reduce duplication. When you write, it's not as simple. First off, in code you will generally have encapsulation, which provides you with the means of limiting the scope of the duplicate logic. With text, your context (encapsulation) often comes from the text prior to it.
One option to deal with this problem might be to extract ideas and then rewrite them in order to make them independent of their previous context. This will help making it possible to read any part of an article without having read what was said previously. However, this will also have the adverse effect of making things much more difficult to follow as there will appear to be no connection between the different sections.
Maybe a better approach to the problem is to think of how one makes code more reusable. One of the biggest issues with code is often that multiple levels of abstractions are used within the same method, which makes reasoning about the method more difficult.
- Should one limit the size of an idea, in the same fashion one tries to limit the number of lines or statements in a function?
- How can one track progress? Is there progress without explicitly specified goals?
- Dump everything in my brain in text documents
- Schedule things that need to be scheduled/reminded
- Decompose tasks and define time/effort estimate
- Write everything down in markdown documents
- Write thoughts in Google Keep while on the move, then merge them in existing documents
- Attempt to manually aggregate content
- No time tracking/evolution of articles
- Manual aggregation is difficult and time consuming
- Most of what is written is never reviewed and potentially forgotten
- Content is not consumed/reused
- Most of the time, the point is to make explicit what is in the brain, so that it can also be referred to later on in a reliable format
- Notes I leave myself aren't tagged
- Get feedback on what I wrote that is consumable by others
AI/AGI/ML - A short overview
History / Edit / PDF / EPUB / BIB / 3 min read (~580 words)After having spent about 200 workdays of my life getting familiar with the fields of AI/AGI/ML, I think it is a good moment for me to reflect back on my learning experience.
Of the three domains, AGI is the least well defined. It has to do with general intelligence, but our definition of intelligence is still up for debate. Furthermore, the different known approaches in the field have different opinions regarding what sort of agent an AGI system should be. Should it be a self-contained entity with its own goals and purpose? Or should it be a system that is exhibiting human capabilities while being the servant of a human being or a group of human beings?
Next is AI, which encompasses many different types of approach to try and solve high level goals such as deciding, planning, organizing,, understanding, etc. AI is about the discovery of strategies to solve problems without having to provide the whole solution. I think it is also where goals for the ML field have been defined.
Finally, ML is about using statistical approaches to solve problems. These problems have to be formalized and specify their input/output in the shape of data, may it be as text, audio, video, or simply vectors of numbers. Most of the recent and exciting development that has happened in ML is due to deep learning, which is the ability to develop deep neural networks that can learn to accomplish certain tasks without writing any explicit logic. This logic would be discovered by the network itself, with some help of the developer which would define a composition of modules that would process the data in order to create associations between the input and outputs.
When I initially started my "adventure", I was interested in AGI. I wanted to know how a machine would be able to exhibit human behavior without having to tell it too much about it. I wanted to learn how you could teach a machine as you would teach a child. The AGI field itself does not currently have any curriculum as to what you should learn about, nor a reference book you can refer yourself or others to (when asked "What should I first read if I want to learn AGI?"). One can be interested in anthropomorphic aspects such as memory, intelligence, perception/senses, processing and so on, while others may be more interested about a computational oriented approach, such as the algorithms, the implementations, the computational complexity of diverse approaches and so on. I think it makes AGI a very interesting topic/field, however it lacks the definiteness of the other fields. One can see this by reading the various iterations of the book "Artificial General Intelligence", which is a collection of articles by various researchers that is generally published every year in relation to its conference.
When I started learning about AGI, I decided I would explore various venues and determine the approaches that matched the most my own interests and which looked the most promising. I looked at different venues such as cognitive science, set theory, logic, universal artificial intelligence, biology, and philosophy amongst others.
Along the road, AGI research has challenged many of my beliefs as well as how I see life. For those two reasons alone, I'm grateful I have been able to dedicate some of my time on it.
One thing I've been wondering about lately is when should one write blog articles (short articles ranging from 250 to 1000 words) or articles (1000+ words) which would be updated (in)frequently but not be as easily consumable.
The main reason I've preferred articles over blog articles is that it makes it possible to keep articles "alive", while once a blog article has been published, it will not evolve over time. By the time it is posted, it is effectively dead and has very little reason to change.
Being a programmer, I've always enjoyed seeing software project evolve over the years. They start small and grow big and complex over time. I like the satisfaction of seeing something grow step by step, and because of that, I've liked the idea of writing about specific topics and seeing these topics and articles change over time.
The problem with articles however is that it is difficult for visitors to consume. Some articles might grow large and complex. Furthermore, it is difficult for visitors to notice any changes from the last time they've come onto the blog.
Another thing is that articles allow the author to "refactor" what he has written, or rewrite/rephrase/reword, such that the article itself improves over time. With a blog article, it is still possible to do so, but the expectation is that a visitor that has already seen the blog article will not re-read it. And here's the important difference between the two: I expect blog articles to be consumed once, but articles to be consumed multiple times. This expectation is likely wrong. The only person that is likely to consume the same article over and over is its author. Visitors may come to the blog many times, but they are unlikely to expect articles to change over time. An extreme example of this idea would be Wikipedia. People will consult the page of the topic they are interested in, but once they've read it, they aren't likely to come back to it again. The only reason they might do so is if they contribute to it.
What this means is that it is still acceptable to improve both blog articles and articles. The difference lies in how we expect our audience to consume each ones. Articles are likely to be seen as references while blog posts are likely to be perceived as an ephemeral observation. One should expect both to be consumed at best once by a visitor, and that the visitor will not come again to see if it has changed.
Thus, in the end, what matters is what we want to communicate with our audience. Do I want to tell you about a thought (a blog article) or do I want to tell you about some information that is likely to evolve over time (an article).
In the case of my machine learning and artificial general intelligence research, what makes sense is to communicate discoveries or shifts in approaches through blog articles while using articles as a location for thoughts to be worked on.
- Have a question
- Ask LLMs
- Identify the field that would best answer this question
- Search online
- Search for articles and books on the topic
- Determine the likelihood that this field has already been researched