Bash is a Unix shell or command line. There are many shells available but Bash is old and standard. None of the alternatives are better, so stick with Bash.
Bash is the first programming language that one should learn. This isn't the conventional approach, but think of human languages like English. Does one start a new language by having conversations or by writing essays? The command line is like having a conversation with one's computer while a program is like an essay. Using the command line will naturally get one used to the right kind of thinking needed for programming. The next step is to write programs (shell scripts) in Bash to automate what one does from the command line. Another benefit to learning bash is that it helps one understand one's computer. Bash deals with basic concepts like files and directories. Note that the first book recommended here is The Unix Programming Environment which introduces the shell. To learn Bash, see my Bash tutorial.
The right environment for programming is the command line and a text editor, not an IDE. IDEs are one of the horrors of modern programming. They are bloated, overcomplicated, and inflexible. Reactionary programming should be based on using simple flexible tools like the shell and a text editor.