If you want to increase ram space for red5 server in Linux just follow below steps.
 The Red 5 server installed location is /usr/local/red5, so open the configure file “red5-hightperf.sh”

local@host# cd /usr/local/red5

vim red5-highperf.sh

Find and replace :

  minimum -  -Xms512M
  maximum - -Xmx1024M

replace the ram space minimum and maximum, Once updated in file, save the file with this :wq!
You want to restart the RED5 Server or directly kill the process by kill command  or pkill command and run again.
 
local@host# pgrep -f java
12352
local@host# kill -9 12352

or directly kill the red5 server by pkill -9 java.
local@host# sh red5.sh &

local@host# pgrep -f java
1365

Check the red5 status  with pgrep command now, the red5 PID is 1365.