Try selecting all the Duration cells, then right-click in a cell and select
Task Information...
You should have a box next to the duration called "Estimated" which has a
gray box in it. If you click here and empty the check box it should remove
all the question marks.
It seems that you unfortunately can't just select the entire Duration
column, but you can click the first task's duration cell and shift-click the
last task's duration cell to select all the tasks in between.
-Jason