To delete a task just click on the “actions” icon on its right, then “trash”. You will have to confirm this action.
In case you want to undo this and move the task out of the trash, just go to "Trash" (4th icon at the top of the project organizer, so left of the page, close to the main menu of the application).
There you just need to select “restore” by clicking on the “actions” icon on the right of the task name.
Note that when projects just have one task and you move them to Trash, the whole project goes there (so no empty projects are allowed in the project tree). We did it like this to avoid having empty items around, but let us know if that bothers you!