The Lottery and Other Stories by Shirley Jackson
After You, My Dear Alphonse (read it here)
I was curious about the title. Was it a common cultural use? Apparently so. According to this resource, it "originated with a comic strip that first appeared in 1905. The expression is generally used when two people go back and forth, suggesting the other go first, as a way of being polite."
I know I've read this story before as well.
Here's another case in which the children exist in a world that's completely inaccessible to the adult. The two are simply playing, trading the saying, carting props around, and looking for dinner. Mom, on the other hand, assumes her child ordered Boyd, the Negro boy, to haul wood, assumes Boyd would need hand-me-down clothes, and didn't feel she could assume Boyd's father held a job. When Boyd has "plenty of clothes, thank you." Mrs. Wilson thinks him ungrateful and is angry, or rather, disappointed. He hasn't fulfilled her image of a Negro boy.
Again the ending completes the separation between the children and the adult:
"Is your mother still mad?" Mrs. Wilson heard Boyd ask in a low voice.
"I don't know," Johnny said. "She's screwy sometimes."
"So's mine," Boyd said. He hesitated. "After you, my dear Alphonse."