Saturday, June 8, 2013

Color image segmentation by thresholding with Java

You may need the image segmentation tool for various image analysis tasks. Different tasks may also need specialized segmentation algorithm, for example the segmentation process may discard all green pixels and keep yellow and red, or if you know the object you are trying to detect contains only blue.

It may be difficult to adjust one general segmentation algorithm for special cases, so you may need to tweak the segmentation code itself, not just the parameters.

I will present here one variation from the simplest segmentation algorithm, segmentation by thresholding, implemented in pure Java.

More sophisticated methods exist, and you should use them if your problem requires so.

In my opinion, this thresholding function gives you a nice starting point when starting to make experiments with segmentation. From the command line, you can only adjust the threshold level, int value 0 - 255, but from the source you can change the connectivity testing and balance the color thresholding if needed.

Example images:
The original photo
Segmented with threshold value 20
Segmented with threshold value 40
Segmented with threshold value 60
Segmented with threshold value 80
Segmented with threshold value 100
Segmented with threshold value 120


Feel free to copy the code and adapt it to your own source and frameworks.

Improvements:
I will make an version from the algorithm where the segment color is averaged from the pixels in the segment. In this version, the color is the color which is the "seed" pixel color.

The algorithm in pseudo code:

for each pixel in image
    if pixel is not in segment
        create new segment
        add the pixel as new "seed" point to list of candidates
        while segment has candidate points
            remove first point from the list of candidates
            if the first candidate point is within threshold limit 

                add the first candidate point to the segment
                add neighbor pixel above to the candidate list
                add neighbor pixel below to the candidate list
                add neighbor pixel right to the candidate list
                add neighbor pixel left  to the candidate list


This kind of algorithm must use the candidate list and while loop instead of recursion, because in the worst case, the whole image (its pixels) belong to one segment. In that case, when using recursion, you will most likely see "StackOverflowError" with big images.

The Java implementation:


package popscan; 
import java.awt.image.BufferedImage
import java.io.File
import java.util.Arrays
import java.util.Vector
import javax.imageio.ImageIO
public class Segmentize { 
    // value for visited pixel 
    int VISITED = 0x00FF0000;         
    // value for not visited pixel 
    int NOT_VISITED = 0x00000000;    
    // source image filename 
    String _srcFilename;     
    // destination image filename 
    String _dstFilename;     
     
    // the source image 
    BufferedImage _srcImage;     
    // the destination image 
    BufferedImage _dstImage;     
     
    // threshold value 0 - 255 
    int _threshold;         
    // image width 
    int _width;             
    // image height 
    int _height;         
    // "seed" color / segment color 
    int _color;             
    // red value from seed 
    int _red;             
    // green value from seed 
    int _green;             
    // blue value from seed 
    int _blue;             
     
    // pixels from source image 
    int[] _pixels;         
    // table for keeping track or visits 
    int[] _visited;         
    // keeping for candidate points 
    Vector<SPoint> _points;  
     
    class SPoint { 
        int x; 
        int y; 
        public SPoint(int x, int y) { 
            this.x = x; 
            this.y = y; 
        } 
    } 
     
    public static void main(String[] args) { 
        if (args.length!=3) { 
            System.out.println("Usage: java Segmentize
                                + " [source image filename]
                                + " [destination image filename]
                                + " [threshold 0-255]")
            return
        } 
        // parse arguments 
        String src = args[0]
        String dst = args[1]
        int threshold = Integer.parseInt(args[2])
         
        // create new Segmentize object 
        Segmentize s = new Segmentize(loadImage(src),threshold)
        // call the function to actually start the segmentation 
        BufferedImage dstImage = s.segmentize()
        // save the resulting image 
        saveImage(dst, dstImage)
    } 
     
    public Segmentize(BufferedImage _srcImage, int threshold) { 
        _threshold = threshold; 
        _width       = _srcImage.getWidth()
        _height       = _srcImage.getHeight()
        // extract pixels from source image 
        _pixels       = _srcImage.getRGB(0, 0, _width, _height, 
                                        null, 0, _width)
        // create empty destination image 
        _dstImage  = new BufferedImage(_width,  
                                        _height,  
                                        BufferedImage.TYPE_INT_RGB)
        _visited   = new int[_pixels.length]
        _points       = new Vector<SPoint>()
    } 
     
    private BufferedImage segmentize() { 
        // initialize points 
        _points.clear()
        // clear table with NOT_VISITED value 
        Arrays.fill(_visited, NOT_VISITED)
        // loop through all pixels 
        for (int x=0;x<_width;x++) { 
            for (int y=0;y<_height;y++) { 
                // if not visited, start new segment 
                if (_visited[_width*y+x]==NOT_VISITED) { 
                    // extract segment color info from pixel 
                    _color = _pixels[_width*y+x]
                    _red   = _color>>16&0xff; 
                    _green = _color>>8&0xff; 
                    _blue  = _color&0xff; 
                    // add "seed" 
                    _points.add(new SPoint(x, y))
                    // start finding neighboring pixels 
                    flood()
                } 
            } 
        } 
        // save the result image 
        _dstImage.setRGB(0, 0, _width, _height, _pixels, 0, _width)
        return _dstImage; 
    } 
     
    public void flood() { 
        // while there are candidates in points vector 
        while (_points.size()>0) { 
            // remove the first candidate 
            SPoint current = _points.remove(0)
            int x = current.x; 
            int y = current.y; 
            if ((x>=0)&&(x<_width)&&(y>=0)&&(y<_height)) { 
                // check if the candidate is NOT_VISITED yet 
                if (_visited[_width*y+x]==NOT_VISITED) { 
                    // extract color info from candidate pixel 
                    int _c = _pixels[_width*y+x]
                    int red   = _c>>16&0xff; 
                    int green = _c>>8&0xff; 
                    int blue  = _c>>0&0xff; 
                    // calculate difference between  
                    // seed's and candidate's 
                    // red, green and blue values 
                    int rx = Math.abs(red - _red)
                    int gx = Math.abs(green - _green)
                    int bx = Math.abs(blue - _blue)
                    // if all colors are under threshold 
                    if (rx<=_threshold 
                            &&gx<=_threshold 
                                &&bx<=_threshold) { 
                        // add the candidate to the segment (image) 
                        _pixels[_width*y+x] = _color; 
                        // mark the candidate as visited 
                        _visited[_width*y+x] = VISITED
                        // add neighboring pixels as candidate 
                        // (8-connected here) 
                        _points.add(new SPoint(x-1,y-1))
                        _points.add(new SPoint(x  ,y-1))
                        _points.add(new SPoint(x+1,y-1))
                        _points.add(new SPoint(x-1,y))
                        _points.add(new SPoint(x+1,y))
                        _points.add(new SPoint(x-1,y+1))
                        _points.add(new SPoint(x  ,y+1))
                        _points.add(new SPoint(x+1,y+1))
                    } 
                } 
            } 
        } 
    } 
     
    public static void saveImage(String filename,  
            BufferedImage image) { 
        File file = new File(filename)
        try { 
            ImageIO.write(image, "png", file)
        } catch (Exception e) { 
            System.out.println(e.toString()+" Image '"+filename 
                                +"' saving failed.")
        } 
    } 
     
    public static BufferedImage loadImage(String filename) { 
        BufferedImage result = null; 
        try { 
            result = ImageIO.read(new File(filename))
        } catch (Exception e) { 
            System.out.println(e.toString()+" Image '" 
                                +filename+"' not found.")
        } 
        return result; 
    }     
     
} 

1 comment:

Prafull Vernekar said...

Great work, but I got two diffferent versions, the first retains the original image, the next changes the output color.

Any ways useful to me...
I appreciate.