If you’re struggling at formal thinking, coding will be hard for you. I’m not talking about designing software with higher concepts, like thread, network IPC, web application routing, gui,… but more simpler one like basic data structures (list, tree, maps, graphs,…), algorithms (search, sort, balancing trees,…), and paradigms (procedural, functional, oop, relational,…).

I’ve met a lot of programmers where those concepts where only words and not something they have understood. A snippet of code is either something they have to learn or copy, it’s not something they can fluently manipulate. It’s the difference between having to use a dictionary and sample phrases and speaking the language fluently. The former is a chore, while you don’t even notice the latter.

▲

jdw64 4 hours ago | parent [-]

I resonate with much of what you have pointed out, but I have a different perspective on certain parts.

Software consists of various layers, and everyone has their own specific areas of strength. For instance, because I am an application programmer, I often need to write code that prevents the program from halting—aligning with recent programming trends that involve preserving the computation context using monads. In other words, my strengths lie in the overall architecture and interface design. This fundamentally relates to cohesion and coupling. I excel in this area, particularly when dealing with codebases around the 60,000-line mark. This is a realm where books like Clean Code are quite effective (many people dislike it, but it is actually a well-written book). Put differently, I possess the ability to mass-produce software (regardless of absolute quality). Over the course of 7 years, I have built CRUD applications for 43 companies across 16 different domains (ranging from drones and golf simulators to tax SaaS and supermarket POS systems). Therefore, I believe I have at least an average, solid capability in this regard. The primary area where I actually made money was PLC, so while I may not have deep academic expertise, I certainly do not think I lack capability.

In Korea, the profession known as SI (System Integration) is a field where you enter contracts on a "project" basis. In that environment, I have encountered a wide variety of people. From those experiences, my takeaway is that programming is divided into quite several distinct layers.

To speak of algorithms first: I learned basic algorithms and fundamental data structures in university. However, in the field where I worked, there were many people who struggled to implement those basic algorithms, yet they still built a large number of applications. Why is that? There was even someone who made an amount of money I could never dream of touching in my lifetime. Why did he make so much money when he didn't even know how to implement basic algorithms?

The answer is quite simple. It is because the "implementation model" and the "contract and cost model" are different. The contract and cost model is a self-contained body of knowledge. It is the ability to know exactly where to fit a given piece into the puzzle. Implementation is simply the ability to build that piece from scratch. In fact, mostly due to issues like employee turnover and the organization's future maintenance capabilities, many teams (specifically, organizations with lower implementation capabilities) decide on an open-source library and design their architecture based on its API. In these cases, the primary technical challenge becomes how to connect those components based on the performance of that library.

Yet, people tend to think that only those who can implement from scratch are capable of programming. A person who can take someone else's implementation and piece it together to fulfill their own contract is also a programmer, but people frequently forget this. Depending on which layer you exist in, certain knowledge requires you to implement it yourself, while other knowledge only requires you to understand the contract. I believe this is the core of programming.

Those on the side that must design and build libraries or frameworks naturally have things they must know about implementation, as they are creating the SDKs. However, I have seen quite a few cases where these very people have no idea how their work is actually utilized in the upper layers. And these types of knowledge are highly fragmented.

In my case, I am familiar with quite a few paradigms. On my personal homepage wiki, I can differentiate between OOP, DOD (Data-Oriented Design), and others, and in the context of relational databases, I know exactly where the ORM impedance mismatch occurs. However, this is largely an area intertwined with architecture, and its essence is closely linked to David Parnas's theory of information hiding. In other words: to what extent do we hide the internals, and where do we expose them to minimize the contact surface area and ensure a safe connection?

For example, I can't implement PostgreSQL's B-tree. But I can design a business system using PostgreSQL. I only know the name of TCP congestion control—I don't know how to implement it. But I can build networked applications.

That's what an 'industry' really is. The ability to trust the contracts of other people's implementations and assemble them. The ability to trust others.

In that regard, I agree with many of your points. However, I actually believe the software industry needs more of those "average" people. I think the very definition of an "industry" should premise that average people can maintain their livelihoods simply by dedicating themselves to a single specific field. From that perspective, when building one's expertise within this fragmented landscape of knowledge, it is perfectly natural not to know much outside of your own specific layer.

p.s https://www.makonea.com/en-US/casual/cargo-cult-programming-...

▲

skydhash 2 hours ago | parent | next [-]

Moving up and down the layer of abstraction is an orthogonal skill. The skill I’m talking is taking some axiomatic concepts (can be opcodes or functions from a high level DSL) and then build something using the given rules. I took basic algorithms as an example because they are trivial in the grand scheme of things while learning something like flask or Laravel involves too many concepts.

At the core, both involve the same formal thinking, but it’s easier with higher level concepts to have tangible results (it helps with the rote learning) than purely abstract ones.

	▲	jdw64 42 minutes ago \| parent [-]
		After reading your follow-up comment, I realize I may have misunderstood some parts, but I think there is a slight difference in our fundamental perspectives. Still, given your context and your experience, your view is probably the right one for you, and given my worldview and my situation, mine is probably right for me. I agree with your points, and thank you for taking the time to comment. Even though our views differ in some ways, it's precisely because of those differences that we have these conversations—and thanks to your comment, I was able to organize my own thoughts. Have a great day. I always appreciate your thought-provoking comments

▲

Jach an hour ago | parent | prev [-]

I really enjoyed reading your comment. I was reminded of a perspective from Patterns of Software, a collection of essays from the 90s by Richard Gabriel. The very first one talks about the ideal of "reuse".

"A similar idea [to libraries] is the Collected Algorithms published by ACM years ago in an ALGOL-like publication language. I remember when I was a kid in 1968 looking up algorithms for sorting and searching in my first programming job. However, what every manager learns is that reuse under these circumstances requires a process of reuse or at least a policy. First, you need to have a central repository of code. ... Second, there has to be a means of locating the right piece of code ... It does no good to have the right piece of code if no one can find it. Classification in the world of books, reports, magazines, and the like is a profession, called cataloging. Librarians help people find the book. But few software organizations can afford a software cataloger, let alone a librarian to help find the software for its developers. This is because when a developer manager has the choice of hiring another developer or a software librarian, the manager will always hire the developer."

He talks about a few reasons for why this is, one of them being simply that plenty of code for your project will just have to be new / different enough from existing code, so there's not much reuse benefit there. He qualifies that with performance concerns, but that's just one reason why it would have to be different.

Another thing I was reminded of, from the intro of Etudes for Programmers:

"Programming is a craft, and programmers must attain a standard of craftsmanship. Much programming is done in cottage shops -- that is, in small shops with meager tools, much work done by hand, and learning attained from other laborers, by chance, and often not at all. ... the academics have also discovered that a craft cannot be taught well by teachers alone; the guild apprenticeships had considerable merit. In a classic apprenticeship the candidate spent many years doing menial tasks while absorbing fundamental techniques of the trade from more experienced workers in the shop. Gradually the apprentice was given more technical responsibility and, after a formal test of skills, eventually became a journeyman certified competent for all ordinary jobs in the trade. The journeyman traveled the world and, if the muses allowed, one day presented a masterpiece to the guild and became a craftsman of the highest rank -- a master of the guild."

Seeking for the average to be that of journeyman, rather than apprentice, seems like a worthwhile goal for the industry. You sound like a journeyman to me, and I can generally trust other journeyman programmers. We don't all need to be masters to have an industry, and that's probably not possible anyway. You say you can't implement TCP congestion control -- but you can build networked applications. Building networked applications is an ordinary job of the trade, and I'm sure you know how to do it in a number of ways. I also suspect that if you had to create some custom UDP protocol for something, you would be able to implement some form of congestion control for it in not too much time -- even if that involves searching for and copying a known algorithm, or if you're lucky finding and using a pre-built library flexible enough for your custom needs. That highlights two more reasons for why we don't hire librarians over developers. First is that, if you're at a journeyman level, then you can be trusted to handle the rather common scenarios where you have to do something for which there's no perfect matching library or framework or SDK you can just use. Whether you have to roll up your sleeves and go low level, relying on very little, or just need to be able to tie enough separate preexisting things together with some minimal glue to handle the new/different stuff, they're both activities that require a programmer rather than a librarian. (I guess this is just further elaboration of the already mentioned "there will be custom code not eligible for replacement by something reusable" reason.) Second, the task of a librarian to find software has been made much easier since the 90s with things like search engines, massive communication channels to share and find out about stuff, and massive open source repositories. It's so easy now that developers themselves can do a decent job at it among their other duties.

	▲	jdw64 28 minutes ago \| parent [-]
		To me, one of the most agonizing parts of programming is that whenever I think I have discovered something new and write down a good explanation for it, I inevitably find out that a pioneer programmer from long ago has already given the exact same phenomenon a much more elegant name. Your analogy of the Journeyman explains the model I was trying to describe far more gracefully. I actually think all these industry issues arise because the ultimate role most programmers aspire to is that of the Master. They want to be the ones designing the rules, frameworks, and architectures that mass produce journeymen. In reality, a department head at a major tech company certainly holds a position worthy of being called a Master. However, I believe being a Journeyman is a thoroughly valuable and respectable end destination in itself. The reason is that once someone becomes a Master, they drift away from the actual field. They start focusing purely on building tools for the journeymen, whereas it is the journeyman who generally remains closest to the actual consumers. I read your comment carefully. Thank you for taking the time to reply, and have a great day. It's been a weekend full of things to think about.