Get bounding boxes for adjacent instances of a single class in image

Question

I have a dataset with thousands of music score pages and manually annotated bounding boxes for the individual bars:

My objective is now to train a DNN that should ultimately be able to get these bounding boxes on its own. First idea was to use something like the Region Proposal Network (RPN) from Faster R-CNN on top of ResNet or VGG, but I am unsure if this still works because the "objectness" is rather high for almost each section of the page. Plus the regions are mostly touching each other but rarely overlap. Number of bars is roughly somewhere between 1 and 250 per page.

Additionally, the number of systems (=rows of bars) per page is oftentimes not changing between subsequent pages. This might be a very helpful info that RPN would miss. Maybe introduce some sort of recurrency?

Is there anything out there that would be more tailored to my specific problem? Any advise on a better fitting architecture or further tweaks would be highly appreciated.

EDIT: Some more extreme examples:

Answer 1

My first thought would be not to full deep learning on this - It is hard to see but it looks like your regions are bound by vertical lines with many horizontal ones spanning those regions. You can try doing just simple canny filters to detect those lines (maybe with [opencv] - [https://opencv-python-tutroals.readthedocs.io/en/latest/py_tutorials/py_imgproc/py_houghlines/py_houghlines.html]), then find the points where horizontal and vertical lines intersect to form vertical bounds for regions.

Another idea that may help is the sweep-plane algorithm,:[https://scicomp.stackexchange.com/questions/8895/vertical-and-horizontal-segments-intersection-line-sweep] I am just spitballing here, but where notes and horizontals meet will form connected regions. Finding connected regions that contain horizantal lines gets you part of the way. Then slicing those with the output of the vertical line detector (maybe it is a search over similar length groups starting with longest vertical using the tree-based strategy of sweep-plane) is worth a try.

On the lines of the RPN, I have had good experience with SSD for a similar problem (detecting individual drawings on an architectural drawing). SSD differs in that it returns something like 8K proposals with confidences, and then a second pass of tuning the confidence threshold and finding non-overlapping regions got me pretty close, but my intuition says that your dataset is structured enough to have another answer.

I am curious how many pages you have in the dataset. If you have less than a few thousand annotated lets say, it may be harder to train a big neural net, and would lean toward the canny filter/hough transform direction. Also are the pages that are annotated represent a diverse enough sample of the production data?