You are correct in only binding off (cast off) at the start of the row. So, what the pattern is asking you to do is bind off (cast off) the first 4 stitches of each row and work to the end of the row in pattern (bind off (cast off) on both the Right Side [Public] and Wrong Side [Private]) for 8 rows consecutively, resulting in a decrease of 32 stitches. HTH.
Welcome to the group.
Judge: Madame, either you will have to cease knitting or I will have to cease judging. Which shall it be?
Miss Marple: It helps me to concentrate, milord.
Judge: It does not help me, madame.
Miss Marple: Oh. Very well. (puts away knitting)
Judge: Thank you, Madame.
The lyf so short, the craft so long to lerne.
~ Geoffrey Chaucer - The Parliament of Fowls Line 1.