Both you and your neighbour need to agree on boundary fences.