Every play, the offense is required to have 7 players on the line of scrimmage. Typically, you will have 5 of them being players with ineligible numbers (5 Olinemen) and the other 2 with eligible; this is to make things easier on the defense and officiating crew to pick out the 7 players.
New England moved one Olineman off the line of scrimmage and put a RB up there, in his place; they lined him up in the slot though, outside the OTackle to that side, but covered by an outside receiver (thats what made him ineligible). If this isn't picked up (Baltimore didn't), the defense has to account for him in coverage, essentially creating a disadvantage by making it 6 total receivers, rather than 5.
Anyone who knows the rules would have seen that outside receiver and known that the slot receiver was covered and not eligible, regardless of whether they heard the announcement or not (that was Harbaugh's gripe).