It’s been nearly five years since I joined HCL Labs, progressing currently to Associate Director - Research. In that time I’ve been involved in:
- setting up HCL’s Open Source Project Office
- researching the state of rich text editing on the web (as I covered in a session at Collabsphere in 2020)
- leading the modernisation of language, extensions and tooling of LotusScript as VoltScript
- integrating VoltScript into Volt MX Foundry as a first-class language adapter
- web components
- and a variety of other projects
This has covered some degree of coding in a variety of languages and frameworks:
- Java
- Vert.x
- Node-RED
- Markdown
- Node.js
- React
- AngularJS
- LotusScript and VoltScript
- JavaScript
- HTML, CSS, SCSS
- Ruby
- Rust
But there are significant differences between traditional product-focused development and development for research. And it’s useful to highlight those differences, because it takes a certain mind-set and skill-set.
Language skills
As a professional developer, the aim is to gain expertise with whatever language and/or framework is relevant for the job and career you desire. Some languages remain popular for a long time and a developer who began with C/C++ or Java could go throughout their career without using another language. However, frameworks have a shorter lifecycle, so a Java developer may have worked with a number of Java frameworks throughout their life, but something like Spring may take up a large proportion of a professional developer’s career. JavaScript has changed much more in a shorter period of change, and someone who began with JavaScript in the 2000s may already have spent years coding with Dojo, then jQuery, AngularJS and React. ANd that JavaScript may now have morphed into TypeScript.
However, as a researcher, the need is different: rather than expertise in a specific language and/or framework, more important is the ability to implement or extend whatever technology is required. One project may require extending a Java product. Another may require manipulating a React UI with some functionality. A third may require changing both the JavaScript front-end of an application and its Java backend - there is not the luxury of having multiple researchers splitting the work. Yet another may demand that you look at open source options to achieve a specific requirement, and restricting options to specific languages means you cannot properly evaluate the merits of all options. And there may not be the luxury of multiple researchers to “divide and conquer”. The ability to be a “jack of all trades”, to know “enough” and get something working regardless of language or framework is crucial.
Adaptability
This is one aspect of adaptability - the ability to use a variety of languages and frameworks, to know enough to get something working and know who to talk about for extra expertise. But it’s more than that. Research is a lot about being agile, embracing the challenge rather than being paralysed by fear and diving in. Once you’ve started, another key aspect is identifying multiple possible approaches, getting something working quickly and, if you hit a dead-end, identifying the cause so you can resolve it or pivoting quickly to another option.
Many research projects I’ve been on have been no more than a matter of weeks, build a proof of concept quickly that can be demoed and handed on. So speed is important, tapping into expertise where required to clear blockers, and having excellent troubleshooting skills to solve problems. Even if you can’t identify the actual cause, being able to rule out options and reduce the potential causes quickly avoids mental paralysis and exhaustion.
Understanding
At the heart of troubleshooting is understanding. It’s not about knowing the languages in-depth or being an expert. It’s about understanding how the technologies work, knowing where to look when things go wrong, being able to see what worked, knowing what tooling to use to troubleshoot the technology, and knowing how to use alternative techniques to pinpoint where in the process it went wrong.
And this is the most crucial aspect: understanding that everything in IT is a process. A REST service call is a request, which often uses a specific library, travels over the HTTP protocol, hits a server, which receives and processes the request, then sends a response, which again must travel back over the HTTP protocol, is received as a response and processed by code. There are a variety of places where it can go wrong and a variety of places where you can verify how far it got before the problem occurred. Using Postman or another language can verify if the problem is in the code making the request or somewhere else. Using a Swagger spec on the relevant website and browser developer tools can allow you to check what it should look when it works. Understanding how to troubleshoot and verify is crucial to effective research development.
Moreover being able to read the existing code and work out what it’s doing is key. Sometimes documentation may be lacking or the process may be very convoluted. Being able to track a complex process through a lot of code is important. And in order to do so quickly, you need to be able to identify what you need to understand and what you don’t, and you need the willingness and mental discipline to ignore what you don’t need to know. Research does not afford the time to understand everything in the code or get something working in “the right” way.
Being able to understand how things work can also help you see the parallels between one technology, language or framework and another. This can help you get to understand them more quickly and maybe solve problems without deep knowledge - because the same approach worked somewhere similar.
And when you need additional expertise, a good research developer will ask the questions to fill their knowledge gap rather than just get an answer. It’s about learning how to know, not learning per se.
“I have a plan…I have part of a plan
The other aspect of understanding also builds on the knowledge that everything is a process. There is a requirement to achieve a specific outcome. The approach for a research developer is to break that into the required steps. Then it’s a case of identifying possible solutions to achieve each step in the process.
But it’s a mistake to need to need to solve the whole problem up front. Part of a plan is enough, but preferably more than 12%! One of the skills I value most in myself is that I think throughout the development process. The solution is constantly evolving and I may say or read something or some code that triggers an idea for solving part of the process.
Two other important aspects here: firstly, I mentioned identifying possible solutions, plural. That’s important, because there may be more than one way to achieve a particular step. Identifying alternatives allows you to choose the best option, which may be prioritised by speed, language, familiarity or a number of other deciding factors. A prototype may deliberately choose an approach that needs to change when it’s taken to production, and there are a number of scenarios where that’s valid. But knowing it’s a step in the process and understanding alternatives is important to handing it over to someone else.
Secondly, the plan should not be linear. It’s perfectly valid to have a plan for most steps but be missing solutions for a step in the middle. As long as you understand what needs to feed into it and what needs to be fed out of it, you can always solve that problem later on. Tackling the process in a linear fashion is more likely to go down a dead-end and spend more time before you realise it. Looking at the elephant from both ends may make it easier to meet in the middle.
Prototyping
A plan is good. But code is better. In HCL Labs we are always encouraged to write code and show running code rather than slides, even if what is shown is not complete. One notable example I remember is the VoltScript Testing Framework. I began creating a spec for it, but quickly abandoned that to dive into coding. I had an idea of what I wanted it to do, I had enough of a plan to try something. Three and a half days later, I had a fully working prototype of the whole framework demoed on our weekly sprint, and it’s only needed a few small enhancements since.
Getting a prototype working has a number of benefits. It helps solidify ideas, turning theory into proof. It helps identify areas that can be or need to be improved. And it helps others visualise what a solution will look like, which may be better than just a written spec to prompt limitations or enhancements.
Another important aspect here is AI. It is a hot topic currently and using AI-generated code may not be appropriate for product-related work. But research and prototyping is about speed. So use of AI is very valid. But again we come back to understanding - it’s important to understand the code provided rather than just focus on achieving a specific outcome.
Try it, show it, improve it - NOT perfect it
Building on the previous two sections, the key here is to have an idea what you want to do and try it. That way you find out quickly whether it can work and get a better idea of its strengths or weaknesses. Build something, show it, get clarity if you’re on the right track, and iteratively improve it. A lot of aspects may be hard-coded for a start.
And more importantly, quite a bit may be hard-coded when you stop. As a research developer, an important aspect to embrace is that it is not your job to perfect it. It’s not your job to code it in a best practice way. It’s not your job to write all the unit tests, though some tests may speed up development or refactoring of the prototype. And it’s certainly not your job to solve all the problems. Research is done in order to hand on to someone else. But what’s important is to have at least a first pass on problems not addressed and areas for improvement.
Dare to fail
Research is, by its very nature, bleeding edge. It’s about trying to do something that doesn’t have an easy answer, and may not even have a possible answer. It’s important to embrace failure and learn from it, whether that’s hitting a dead end and having to pivot to a different solution to part of a process, or learning that the intended target was unachievable. There is the oft-quoted comment from Thomas Edison that he didn’t fail 700 times, he succeeded in proving 700 ways not to build a lightbulb. Learning how not to do something is still valuable, even if not as valuable as finding out how to do it. And knowing why is was not possible is important, because IT is constantly evolving and that single cause of failure may be resolved or no longer relevant in the future.
Document it
Another important point is that research may not get picked up straight away. So document what it’s doing, document what needs to be considered by whoever comes next, document what you haven’t solved, and document assumptions you’ve made.
As a researcher, it’s not typical to be working on a project for months. It happens, but it’s not the norm. Chances are you’ll be on something completely different, in a different technology and maybe for a totally different project. The documentation may avoid questions or, more importantly, mistaken assumptions or code just used as-is when you intended it to be improved or limitations addressed.
Re-Use it
The other reason for documenting is that the research may not get picked up at all for its intended purpose. But the whole project or some of the learning may become reusable somewhere else later on. I’ve seen a number of cases where research projects have been resurrected for a completely different purpose and, in some cases, be used in that new incarnation in production systems. Be aware of the research that’s done and where some or all of either the learning or the code can be used. Even a failed project may be relevant later on, when circumstances change or technology solves a problem that was previously unsolvable.
Embrace Change
The final important point I’ve learned is to embrace change. It is said the only constant in IT is change. In research, change is constant. The need to pivot at a moment’s notice is crucial, as is the need to work on multiple things at the same time, to be aware of new learning and current trends. The role of research developer is not so much about expertise, it’s about problem solving, about deconstructing and reconstructing.