The main interest of keeping a modification history is to be able to roll back mistakes. For instance:

If the commit is not yet pushed in the remote repository, that's easy. You can reset or revert the modifications.

The first method will delete the commit(s) of the history. It is only valid if you want to remove the last commit(s) but it is the cleanest one. Note that this command can modify the code if you use the option --hard (so be careful when you use it).

Let's image that your mistake is in the middle of valid commits.

In this case, the best option is to use revert to cancel the bad commit. Indeed, this will create an additional commit which will cancel the modifications of the bad commit. Note that you cannot have pending modifications while using this method to make it work.

The corresponding bash command are git reset (documentation) and git revert (documentation).