Brushing against the touchpad during video playback should do nothing, rather than its current behavior of seeking within the video.
Picking up the remote in any orientation and brushing against any part of it during handling, without physically pushing a button, should never result in accidental input. Picking it up should feel safe.
Touchpad seeking during video playback is already available when the video is paused by the Play/Pause button or, intuitively, clicking the trackpad — there’s no need to make it active all the time, when it frequently results in accidentally seeking videos, a highly disruptive usability failure.
This isn’t the Siri Remote’s only problem, but I think it’s the biggest. This rearrangement would solve some other problems.