Jump to content

Cleanup /img/p folder and image folder questions.

masterblaster
 Share

Recommended Posts

masterblaster Rookie

masterblaster

Posted
Posted

Hello,

I've recently updated my shop from 1.4 to 1.5.
 
When I was using 1.4, I loaded tons of products with CSV-import.
 
During the upgrade I realized that even though the shop have just over 30,000 products,  the img/p contained hundreds of thousands of files, some of which were associated with products deleted from the site, surely because of various imports and batch product deletion.
 
My images were originally in "1.4" filesystem ( img/p/x-y.jpg)
 
After using the PS "move" tool to move the to the new img/p/x/y/z/... filesystem and regenerating thumbnail , I see I still have over 100.000+ images in img/p/ root directory. 
However, the "move image" said that all images were moved succesfully. 
Having so many products, I do not know if it's true or not, because I did not have the possibility to check for all product pictures. I think/hope that the "move images" feature rely on database ps_img table to check what are the "right" images to move... Am I right ?
 
Questions :
 
- Is it safe to delete those img/p/...jpg  images (they are eating up a lot of space) ?   Are those "orphans" images or should I keep them on the server ?
- Is there a way or a tool to cleanup unused images from ftp in order to keep img/p clean and get rid of "unlinked" images ?
 
Any suggestion is really appreciated.
 
 
  • Like 1
Link to comment
Share on other sites
  • 2 months later...
convasgr40 Newbie

convasgr40

Posted
Posted

 

Hello,

I've recently updated my shop from 1.4 to 1.5.
 
When I was using 1.4, I loaded tons of products with CSV-import.
 
During the upgrade I realized that even though the shop have just over 30,000 products,  the img/p contained hundreds of thousands of files, some of which were associated with products deleted from the site, surely because of various imports and batch product deletion.
 
My images were originally in "1.4" filesystem ( img/p/x-y.jpg)
 
After using the PS "move" tool to move the to the new img/p/x/y/z/... filesystem and regenerating thumbnail , I see I still have over 100.000+ images in img/p/ root directory. 
However, the "move image" said that all images were moved succesfully. 
Having so many products, I do not know if it's true or not, because I did not have the possibility to check for all product pictures. I think/hope that the "move images" feature rely on database ps_img table to check what are the "right" images to move... Am I right ?
 
Questions :
 
- Is it safe to delete those img/p/...jpg  images (they are eating up a lot of space) ?   Are those "orphans" images or should I keep them on the server ?
- Is there a way or a tool to cleanup unused images from ftp in order to keep img/p clean and get rid of "unlinked" images ?
 
Any suggestion is really appreciated.

 

I have a similar problem. Any suggestions?

Link to comment
Share on other sites
  • 1 month later...
  • 4 months later...
  • 5 months later...
shopimport.nl Newbie

shopimport.nl

Posted
Posted

I've prepared a script.


 


The script will read the img/p folder and check for each file if it exists in the database table ps_image.


In case it doesn't exists then it prints the file and deletes the file. I commented the unlink() so you can run it without really deleting and when you are confident then you can uncomment the "unlink()" command and it will really delete the image files. 


 


You need to configure the root of the shop path. And you can set a max number of files that the script will read from the img/p folder. 


 


The script is not yet prepared for the new img/p/1/2/3/ structure, only for the structure to have all jpg files directly in the img/p. It should be relative easy to improve this.


 


Copy this script into a file "cleanup.php" in the root of your shop. Then edit the value $shop_root and the run the script by the browser (http://yourdomain.com/cleanup.php) or on the console (login with ssh for the experts). 


 


The query to check the ps_image table will always result in 1 row to ensure that no files will be deleted when there are database errors. So the script is secured for unexpected mysql errors (like overload of mysql server which you don't want to result in not finding the images and thus would remove the image files).




<?php

// root path of the shop

$shop_root='/home/kooozybe/public_html/shop/';

// limit number of image files to check, set to 10 for testing

$limit=1000000;



include $shop_root . '/config/settings.inc.php';

$pdo = new PDO( 'mysql:host='._DB_SERVER_.';dbname='._DB_NAME_, _DB_USER_, _DB_PASSWD_ );

$pdo->setAttribute(PDO::ATTR_ERRMODE, PDO::ERRMODE_EXCEPTION);

$r=$pdo->query('select count(1) cnt from ps_image')->fetch();

echo 'count images database: '.$r['cnt'];


// reset some counters

$cnt_files=0;

$cnt_checked=0;

$cnt_not_found=0;

$cnt_found=0;

$path=$shop_root.'/img/p';

if ($handle = opendir($path)) {

    while ($cnt_files != $limit && false !== ($entry = readdir($handle))) {

        if ($entry != "." && $entry != "..") {

                $cnt_files++;

                $pi = explode('-',$entry);

                if($pi[0]>0 && $pi[1]>0) {

                        $cnt_checked++;

                        if(!checkExistsDb($pdo,$pi[0],$pi[1])) {

                                $cnt_not_found++;

                                echo 'rm '.$path.'/'.$entry."\r\n";

// this will delete files                                unlink($path.'/'.$entry);

                        } else {

                                $cnt_found++;

                        }

                }

        }

    }

    closedir($handle);

}


echo 'files: '.$cnt_files.' checked: '.$cnt_checked.' not_found: '.$cnt_not_found.' found: '.$cnt_found;


function checkExistsDb($pdo, $id_product, $id_image) {

        $r=$pdo->query($q='select \'ok\' ok, (select id_image from ps_image where id_product = '.(int)$id_product.' and id_image = '.(int)$id_image.') id_image');

        $row=$r->fetch();

        if($row['ok']!='ok') die( 'Problem with query, please correct');

        return $row['id_image']?true:false;

}

  • Thanks 1
Link to comment
Share on other sites
  • 2 months later...
Writer Newbie

Writer

Posted
Posted (edited)

I have created script which will delete images by size name.

 

Put php file with below code in your home directory and call it from web browser!

 

So for example if you want to delete all images with size "home_default" add to private $files = array() only this file size:

<?php
require(dirname(__FILE__).'/config/config.inc.php');

new deleteImages();

class deleteImages {

    private $files = array(
        'home_default'
    );
    private $imgDir = 'img/p';

    function __construct() {
        $this->directoryToArray($this->imgDir);
    }

    public function directoryToArray($directory, $recursive = true, $listDirs = false, $listFiles = true, $exclude = '') {
        $arrayItems = array();
        $skipByExclude = false;
        $handle = opendir($directory);
        if ($handle) {
            while (false !== ($file = readdir($handle))) {
                preg_match("/(^(([\.]){1,2})$|(\.(svn|git|md))|(Thumbs\.db|\.DS_STORE))$/iu", $file, $skip);
                if($exclude){
                    preg_match($exclude, $file, $skipByExclude);
                }
                if (!$skip && !$skipByExclude) {
                    if (is_dir($directory. DIRECTORY_SEPARATOR . $file)) {
                        if($recursive) {
                            $arrayItems = array_merge($arrayItems, $this->directoryToArray($directory. DIRECTORY_SEPARATOR . $file, $recursive, $listDirs, $listFiles, $exclude));
                        }
                        if($listDirs){
                            $file = $directory . DIRECTORY_SEPARATOR . $file;
                            $arrayItems[] = $file;
                        }
                    } else {
                        if($listFiles){
                            $file = $directory . DIRECTORY_SEPARATOR . $file;
                            foreach($this->files as $fileD) {
                                if(strpos($file, $fileD) !== false) {
                                    unlink($file);
                                }
                            }
                            $arrayItems[] = $file;
                        }
                    }
                }
            }
            closedir($handle);
        }
        return $arrayItems;
    }
}
?>
Edited by Writer (see edit history)
Link to comment
Share on other sites
  • 3 weeks later...
  • 1 month later...
mkbond777 Newbie

mkbond777

Posted
Posted

With the help of shopimport.nl code I have written code to delete images for the directory structure img/p/1/2/3 as well.

<?php
// root path of the shop
$shop_root='/home/xyz/public_html/';
// limit number of image files to check, set to 10 for testing
$limit=10;
 
 
include $shop_root . '/config/settings.inc.php';
$pdo = new PDO( 'mysql:host='._DB_SERVER_.';dbname='._DB_NAME_, _DB_USER_, _DB_PASSWD_ );
$pdo->setAttribute(PDO::ATTR_ERRMODE, PDO::ERRMODE_EXCEPTION);
$r=$pdo->query('select count(1) cnt from ps_image')->fetch();
echo 'count images database: '.$r['cnt'] . "<Br />";
 
// reset some counters
$cnt_files=0;
$cnt_checked=0;
$cnt_not_found=0;
$cnt_found=0;

for($ii=1; ($ii<=9) && ($cnt_files != $limit); $ii++)
{
	$path=$shop_root.'img/p/'.$ii;
	delImage($path);
	for($jj=0; ($jj<=9) && ($cnt_files != $limit); $jj++)
	{	
		$path=$shop_root.'img/p/'.$ii.'/'.$jj;
		delImage($path);
		for($kk=0; ($kk<=9) && ($cnt_files != $limit); $kk++)
		{
			$path=$shop_root.'img/p/'.$ii.'/'.$jj.'/'.$kk;
			delImage($path);
			for($ll=0; ($ll<=9) && ($cnt_files != $limit); $ll++)
			{
				$path=$shop_root.'img/p/'.$ii.'/'.$jj.'/'.$kk.'/'.$ll;
				delImage($path);
			}	
		}
 	}
 		
}
echo 'files: '.$cnt_files.' checked: '.$cnt_checked.' not_found: '.$cnt_not_found.' found: '.$cnt_found;

function delImage($imageDir)
{
	global $limit, $pdo, $cnt_files, $cnt_checked, $cnt_not_found, $cnt_found;
	if ($handle = @opendir($imageDir)) {			//@ is wriiten to avoid warning message and is handled in else condition
		echo $imageDir."<BR />";
    		while ($cnt_files != $limit && false !== ($entry = readdir($handle))) {
        		if ($entry != "." && $entry != "..") {
                		$cnt_files++;
                		$pi = explode('-',$entry);
                		if($pi[0]>0 && !empty($pi[1])) {
                        		$cnt_checked++;
                        		if(!checkExistsDb($pdo,$pi[0])) {
                                		$cnt_not_found++;
                                		echo 'rm '.$imageDir.'/'.$entry."<BR />";
			        		unlink($imageDir.'/'.$entry);
                        		} else {
                                		$cnt_found++;
                       			}
                		}
        		}
    		}
    			closedir($handle);
	}
	else
	{
		echo $imageDir." doesn't exist".'<BR />';
	}

}
 
function checkExistsDb($pdo, $id_image) {
        $r=$pdo->query($q='select \'ok\' ok, (select id_image from ps_image where id_image = '.(int)$id_image.') id_image');
        $row=$r->fetch();
        if($row['ok']!='ok') die( 'Problem with query, please correct');
        return $row['id_image']?true:false;
}
  • Like 2
Link to comment
Share on other sites
  • 3 months later...
  • 1 month later...
mkbond777 Newbie

mkbond777

Posted
Posted

Hello!

 

mkbond777, I started your php script and results was files: 50000 checked: 27977 not_found: 0 found: 27977, but nothing are deleted? Why? Thank you! Best regards, Denis

 

Hi Denis,

 

From here, it will be really tough for me to find out why this script is not working for you?

 

But you can try a couple of things to fix the issue yourself.

 

1. Try to find out which folder has images which are not present in your db and hence are not being used anymore in your website.

2. If you figure out that folder, then write a echo command in the function checkExistsDb($pdo, $id_image) after $row=$r->fetch(); to figure out whether this function itself is called or not for that folder.

 

Regards

Manish

Link to comment
Share on other sites
  • 3 months later...
JuRaSSiCBoY Newbie

JuRaSSiCBoY

Posted
Posted (edited)

Hi Denis,

 

From here, it will be really tough for me to find out why this script is not working for you?

 

But you can try a couple of things to fix the issue yourself.

 

1. Try to find out which folder has images which are not present in your db and hence are not being used anymore in your website.

2. If you figure out that folder, then write a echo command in the function checkExistsDb($pdo, $id_image) after $row=$r->fetch(); to figure out whether this function itself is called or not for that folder.

 

Regards

Manish

Same here, my Prestashop 1.5.4.0 is growing fastly (now 1,83GB) and I notice if a product is removed, the image related to the product remains at our server. You script shows a lot of lines like:

 

 

...

...

/home/myweb/public_html/img/p/9/9/8

/home/myweb/public_html/img/p/9/9/8/0 doesn't exist

/home/myweb/public_html/img/p/9/9/8/1 doesn't exist

/home/myweb/public_html/img/p/9/9/8/2 doesn't exist

/home/myweb/public_html/img/p/9/9/8/3 doesn't exist

/home/myweb/public_html/img/p/9/9/8/4 doesn't exist

/home/myweb/public_html/img/p/9/9/8/5 doesn't exist

/home/myweb/public_html/img/p/9/9/8/6 doesn't exist

/home/myweb/public_html/img/p/9/9/8/7 doesn't exist

/home/myweb/public_html/img/p/9/9/8/8 doesn't exist

/home/myweb/public_html/img/p/9/9/8/9 doesn't exist

/home/myweb/public_html/img/p/9/9/9

/home/myweb/public_html/img/p/9/9/9/0 doesn't exist

/home/myweb/public_html/img/p/9/9/9/1 doesn't exist

/home/myweb/public_html/img/p/9/9/9/2 doesn't exist

/home/myweb/public_html/img/p/9/9/9/3 doesn't exist

/home/myweb/public_html/img/p/9/9/9/4 doesn't exist

/home/myweb/public_html/img/p/9/9/9/5 doesn't exist

/home/myweb/public_html/img/p/9/9/9/6 doesn't exist

/home/myweb/public_html/img/p/9/9/9/7 doesn't exist

/home/myweb/public_html/img/p/9/9/9/8 doesn't exist

/home/myweb/public_html/img/p/9/9/9/9 doesn't exist

files: 29013 checked: 22290 not_found: 0 found: 22290

 

But every attempt shows the same and seems like It's not removing anything. I've put a "var_dump" and an "echo" at "unlink($imageDir.'/'.$entry);" to see if " if(!checkExistsDb($pdo,$pi[0])) {" block of code is running but the script is not arriving at this point. So the code flux never get into:

 

                            if(!checkExistsDb($pdo,$pi[0])) {
                                                $cnt_not_found++;
                                                echo 'rm '.$imageDir.'/'.$entry."<BR />";
                                                unlink($imageDir.'/'.$entry);

 

 

Thanks a lot for your help and time. 

Edited by JuRaSSiCBoY (see edit history)
Link to comment
Share on other sites
  • 4 weeks later...
ELEGANTAL Rookie

ELEGANTAL

Posted
Posted (edited)

The problem of img/p folder getting bigger and bigger is usually because of Prestashop being unable to delete thumbnail images before it regenerates them.
In the Images page on backoffice there is REGENERATE THUMBNAILS section. When you use that, basically Prestashop deletes old images and create new ones. It usually generates thousands of images. Imagine what happens when it cannot delete old ones.

Edited by PrestaProf (see edit history)
Link to comment
Share on other sites
  • 11 months later...
Nouvelle Vape Newbie

Nouvelle Vape

Posted
Posted

 

With the help of shopimport.nl code I have written code to delete images for the directory structure img/p/1/2/3 as well.

<?php
// root path of the shop
$shop_root='/home/xyz/public_html/';
// limit number of image files to check, set to 10 for testing
$limit=10;
 
 
include $shop_root . '/config/settings.inc.php';
$pdo = new PDO( 'mysql:host='._DB_SERVER_.';dbname='._DB_NAME_, _DB_USER_, _DB_PASSWD_ );
$pdo->setAttribute(PDO::ATTR_ERRMODE, PDO::ERRMODE_EXCEPTION);
$r=$pdo->query('select count(1) cnt from ps_image')->fetch();
echo 'count images database: '.$r['cnt'] . "<Br />";
 
// reset some counters
$cnt_files=0;
$cnt_checked=0;
$cnt_not_found=0;
$cnt_found=0;

for($ii=1; ($ii<=9) && ($cnt_files != $limit); $ii++)
{
	$path=$shop_root.'img/p/'.$ii;
	delImage($path);
	for($jj=0; ($jj<=9) && ($cnt_files != $limit); $jj++)
	{	
		$path=$shop_root.'img/p/'.$ii.'/'.$jj;
		delImage($path);
		for($kk=0; ($kk<=9) && ($cnt_files != $limit); $kk++)
		{
			$path=$shop_root.'img/p/'.$ii.'/'.$jj.'/'.$kk;
			delImage($path);
			for($ll=0; ($ll<=9) && ($cnt_files != $limit); $ll++)
			{
				$path=$shop_root.'img/p/'.$ii.'/'.$jj.'/'.$kk.'/'.$ll;
				delImage($path);
			}	
		}
 	}
 		
}
echo 'files: '.$cnt_files.' checked: '.$cnt_checked.' not_found: '.$cnt_not_found.' found: '.$cnt_found;

function delImage($imageDir)
{
	global $limit, $pdo, $cnt_files, $cnt_checked, $cnt_not_found, $cnt_found;
	if ($handle = @opendir($imageDir)) {			//@ is wriiten to avoid warning message and is handled in else condition
		echo $imageDir."<BR />";
    		while ($cnt_files != $limit && false !== ($entry = readdir($handle))) {
        		if ($entry != "." && $entry != "..") {
                		$cnt_files++;
                		$pi = explode('-',$entry);
                		if($pi[0]>0 && !empty($pi[1])) {
                        		$cnt_checked++;
                        		if(!checkExistsDb($pdo,$pi[0])) {
                                		$cnt_not_found++;
                                		echo 'rm '.$imageDir.'/'.$entry."<BR />";
			        		unlink($imageDir.'/'.$entry);
                        		} else {
                                		$cnt_found++;
                       			}
                		}
        		}
    		}
    			closedir($handle);
	}
	else
	{
		echo $imageDir." doesn't exist".'<BR />';
	}

}
 
function checkExistsDb($pdo, $id_image) {
        $r=$pdo->query($q='select \'ok\' ok, (select id_image from ps_image where id_image = '.(int)$id_image.') id_image');
        $row=$r->fetch();
        if($row['ok']!='ok') die( 'Problem with query, please correct');
        return $row['id_image']?true:false;
}

 

 

With the help of shopimport.nl code I have written code to delete images for the directory structure img/p/1/2/3 as well.

<?php
// root path of the shop
$shop_root='/home/xyz/public_html/';
// limit number of image files to check, set to 10 for testing
$limit=10;
 
 
include $shop_root . '/config/settings.inc.php';
$pdo = new PDO( 'mysql:host='._DB_SERVER_.';dbname='._DB_NAME_, _DB_USER_, _DB_PASSWD_ );
$pdo->setAttribute(PDO::ATTR_ERRMODE, PDO::ERRMODE_EXCEPTION);
$r=$pdo->query('select count(1) cnt from ps_image')->fetch();
echo 'count images database: '.$r['cnt'] . "<Br />";
 
// reset some counters
$cnt_files=0;
$cnt_checked=0;
$cnt_not_found=0;
$cnt_found=0;

for($ii=1; ($ii<=9) && ($cnt_files != $limit); $ii++)
{
	$path=$shop_root.'img/p/'.$ii;
	delImage($path);
	for($jj=0; ($jj<=9) && ($cnt_files != $limit); $jj++)
	{	
		$path=$shop_root.'img/p/'.$ii.'/'.$jj;
		delImage($path);
		for($kk=0; ($kk<=9) && ($cnt_files != $limit); $kk++)
		{
			$path=$shop_root.'img/p/'.$ii.'/'.$jj.'/'.$kk;
			delImage($path);
			for($ll=0; ($ll<=9) && ($cnt_files != $limit); $ll++)
			{
				$path=$shop_root.'img/p/'.$ii.'/'.$jj.'/'.$kk.'/'.$ll;
				delImage($path);
			}	
		}
 	}
 		
}
echo 'files: '.$cnt_files.' checked: '.$cnt_checked.' not_found: '.$cnt_not_found.' found: '.$cnt_found;

function delImage($imageDir)
{
	global $limit, $pdo, $cnt_files, $cnt_checked, $cnt_not_found, $cnt_found;
	if ($handle = @opendir($imageDir)) {			//@ is wriiten to avoid warning message and is handled in else condition
		echo $imageDir."<BR />";
    		while ($cnt_files != $limit && false !== ($entry = readdir($handle))) {
        		if ($entry != "." && $entry != "..") {
                		$cnt_files++;
                		$pi = explode('-',$entry);
                		if($pi[0]>0 && !empty($pi[1])) {
                        		$cnt_checked++;
                        		if(!checkExistsDb($pdo,$pi[0])) {
                                		$cnt_not_found++;
                                		echo 'rm '.$imageDir.'/'.$entry."<BR />";
			        		unlink($imageDir.'/'.$entry);
                        		} else {
                                		$cnt_found++;
                       			}
                		}
        		}
    		}
    			closedir($handle);
	}
	else
	{
		echo $imageDir." doesn't exist".'<BR />';
	}

}
 
function checkExistsDb($pdo, $id_image) {
        $r=$pdo->query($q='select \'ok\' ok, (select id_image from ps_image where id_image = '.(int)$id_image.') id_image');
        $row=$r->fetch();
        if($row['ok']!='ok') die( 'Problem with query, please correct');
        return $row['id_image']?true:false;
}

Many thanks, your script works great!

 

Olivier

Link to comment
Share on other sites
Nouvelle Vape Newbie

Nouvelle Vape

Posted
Posted

Hi can someone confirm this script works on PS 1.6.1.11?

 

I can't make it work, still searching the problem

Hi,

 

The script worked for me on ps 1.6.1.11.

I placed it in the root folder of ps and first commented the line that effectively delete the pictures. I checked with the log which pictures it planned to delete and then uncommented the line. It saved me hundreds of MB!

 

Hope you will be able to make it work.

Olivier

Link to comment
Share on other sites
  • 4 weeks later...
Cătălin Newbie

Cătălin

Posted
Posted

The script isn't working no matter what you do. There is redundancy I don't know.

 

For the very big p folder just change in the Preferences-Images the Format for jpg only, change the JPEG Compression to 50 and Regenerate Thumbnails. It will save 2 thirds of the space. In my case I compressed the p folder from 2.9 G to 0.9 G.

Link to comment
Share on other sites
  • 2 months later...
j.kaspar Newbie

j.kaspar

Posted
Posted (edited)

The script isn't working no matter what you do. There is redundancy I don't know.

 

For the very big p folder just change in the Preferences-Images the Format for jpg only, change the JPEG Compression to 50 and Regenerate Thumbnails. It will save 2 thirds of the space. In my case I compressed the p folder from 2.9 G to 0.9 G.

 

That is not a solution. Compression set to 50, results to a big loss in a quality. The script works well. If You want to delete all the images, including the onriginal one, it is necessary to remove this condition

&& !empty($pi[1])

here

if($pi[0]>0 && !empty($pi[1])) {

Otherwise, it deletes only the generated thumbnails.

Edited by j.kaspar (see edit history)
Link to comment
Share on other sites
  • 1 year later...
  • 2 months later...
ariom Newbie

ariom

Posted
Posted
On 1/23/2015 at 1:41 PM, shopimport.nl said:

I've prepared a script.

The script will read the img/p folder and check for each file if it exists in the database table ps_image.

In case it doesn't exists then it prints the file and deletes the file. I commented the unlink() so you can run it without really deleting and when you are confident then you can uncomment the "unlink()" command and it will really delete the image files.

You need to configure the root of the shop path. And you can set a max number of files that the script will read from the img/p folder.

The script is not yet prepared for the new img/p/1/2/3/ structure, only for the structure to have all jpg files directly in the img/p. It should be relative easy to improve this.

Copy this script into a file "cleanup.php" in the root of your shop. Then edit the value $shop_root and the run the script by the browser (http://yourdomain.com/cleanup.php) or on the console (login with ssh for the experts).

The query to check the ps_image table will always result in 1 row to ensure that no files will be deleted when there are database errors. So the script is secured for unexpected mysql errors (like overload of mysql server which you don't want to result in not finding the images and thus would remove the image files). 


<?php

// root path of the shop
$shop_root='/home/kooozybe/public_html/shop/';

// limit number of image files to check, set to 10 for testing
$limit=1000000;

include $shop_root . '/config/settings.inc.php';
$pdo = new PDO( 'mysql:host='._DB_SERVER_.';dbname='._DB_NAME_, _DB_USER_, _DB_PASSWD_ );
$pdo->setAttribute(PDO::ATTR_ERRMODE, PDO::ERRMODE_EXCEPTION);
$r=$pdo->query('select count(1) cnt from ps_image')->fetch();
echo 'count images database: '.$r['cnt'];

// reset some counters
$cnt_files=0;
$cnt_checked=0;
$cnt_not_found=0;
$cnt_found=0;
$path=$shop_root.'/img/p';

if ($handle = opendir($path)) {
    while ($cnt_files != $limit && false !== ($entry = readdir($handle))) {
        if ($entry != "." && $entry != "..") {
                $cnt_files++;
                $pi = explode('-',$entry);
                if($pi[0]>0 && $pi[1]>0) {
                        $cnt_checked++;
                        if(!checkExistsDb($pdo,$pi[0],$pi[1])) {
                                $cnt_not_found++;
                                echo 'rm '.$path.'/'.$entry."\r\n";

// this will delete files                                unlink($path.'/'.$entry);

                        } else {

                                $cnt_found++;
                        }
                }
        }
    }
    closedir($handle);
}
echo 'files: '.$cnt_files.' checked: '.$cnt_checked.' not_found: '.$cnt_not_found.' found: '.$cnt_found;

function checkExistsDb($pdo, $id_product, $id_image) {
        $r=$pdo->query($q='select \'ok\' ok, (select id_image from ps_image where id_product = '.(int)$id_product.' and id_image = '.(int)$id_image.') id_image');
        $row=$r->fetch();
        if($row['ok']!='ok') die( 'Problem with query, please correct');
        return $row['id_image']?true:false;
}

 

Ullalah ... after more than 4 years your script stil working perfectly on my PS 1.6.1.4  (and its TB fork migration) with old legacy structure image p folder .... going down from 2 GB to 1.1 GB with easy.

Big THANKS to @shopimport.nl for doing the job ... you saved my time  ...and disk space ...!!!!

Link to comment
Share on other sites
  • 4 weeks later...
ballashop Enthusiast

ballashop

Posted
Posted
On 5/31/2015 at 3:57 PM, mkbond777 said:

With the help of shopimport.nl code I have written code to delete images for the directory structure img/p/1/2/3 as well. 


<?php
// root path of the shop
$shop_root='/home/xyz/public_html/';
// limit number of image files to check, set to 10 for testing
$limit=10;
 
 
include $shop_root . '/config/settings.inc.php';
$pdo = new PDO( 'mysql:host='._DB_SERVER_.';dbname='._DB_NAME_, _DB_USER_, _DB_PASSWD_ );
$pdo->setAttribute(PDO::ATTR_ERRMODE, PDO::ERRMODE_EXCEPTION);
$r=$pdo->query('select count(1) cnt from ps_image')->fetch();
echo 'count images database: '.$r['cnt'] . "<Br />";
 
// reset some counters
$cnt_files=0;
$cnt_checked=0;
$cnt_not_found=0;
$cnt_found=0;

for($ii=1; ($ii<=9) && ($cnt_files != $limit); $ii++)
{
	$path=$shop_root.'img/p/'.$ii;
	delImage($path);
	for($jj=0; ($jj<=9) && ($cnt_files != $limit); $jj++)
	{	
		$path=$shop_root.'img/p/'.$ii.'/'.$jj;
		delImage($path);
		for($kk=0; ($kk<=9) && ($cnt_files != $limit); $kk++)
		{
			$path=$shop_root.'img/p/'.$ii.'/'.$jj.'/'.$kk;
			delImage($path);
			for($ll=0; ($ll<=9) && ($cnt_files != $limit); $ll++)
			{
				$path=$shop_root.'img/p/'.$ii.'/'.$jj.'/'.$kk.'/'.$ll;
				delImage($path);
			}	
		}
 	}
 		
}
echo 'files: '.$cnt_files.' checked: '.$cnt_checked.' not_found: '.$cnt_not_found.' found: '.$cnt_found;

function delImage($imageDir)
{
	global $limit, $pdo, $cnt_files, $cnt_checked, $cnt_not_found, $cnt_found;
	if ($handle = @opendir($imageDir)) {			//@ is wriiten to avoid warning message and is handled in else condition
		echo $imageDir."<BR />";
    		while ($cnt_files != $limit && false !== ($entry = readdir($handle))) {
        		if ($entry != "." && $entry != "..") {
                		$cnt_files++;
                		$pi = explode('-',$entry);
                		if($pi[0]>0 && !empty($pi[1])) {
                        		$cnt_checked++;
                        		if(!checkExistsDb($pdo,$pi[0])) {
                                		$cnt_not_found++;
                                		echo 'rm '.$imageDir.'/'.$entry."<BR />";
			        		unlink($imageDir.'/'.$entry);
                        		} else {
                                		$cnt_found++;
                       			}
                		}
        		}
    		}
    			closedir($handle);
	}
	else
	{
		echo $imageDir." doesn't exist".'<BR />';
	}

}
 
function checkExistsDb($pdo, $id_image) {
        $r=$pdo->query($q='select \'ok\' ok, (select id_image from ps_image where id_image = '.(int)$id_image.') id_image');
        $row=$r->fetch();
        if($row['ok']!='ok') die( 'Problem with query, please correct');
        return $row['id_image']?true:false;
}

Hi,

I try in local server but I've got HTTP ERROR 500.

Thanks for any reply

Link to comment
Share on other sites
  • 8 months later...
  • 1 year later...
Jesus Ayala Newbie

Jesus Ayala

Posted
Posted (edited)

I know this post is old. But the script worked for me. Here it is with some modifications to support two more levels of folders.
  

<?php
// root path of the shop
$shop_root='/var/www/html/';
// limit number of image files to check, set to 10 for testing
$limit=100000000;
 
 
//include $shop_root . '/config/settings.inc.php';
$pdo = new PDO( 'mysql:host=localhost;dbname=DBNAME', 'USER', 'PASSWORD' );
$pdo->setAttribute(PDO::ATTR_ERRMODE, PDO::ERRMODE_EXCEPTION);
$r=$pdo->query('select count(1) cnt from ps_image')->fetch();
echo 'count images database: '.$r['cnt'] . "\n";
 
// reset some counters
$cnt_files=0;
$cnt_checked=0;
$cnt_not_found=0;
$cnt_found=0;

for($ii=1; ($ii<=9) && ($cnt_files != $limit); $ii++)
{
        $path=$shop_root.'img/p/'.$ii;
        delImage($path);
        for($jj=0; ($jj<=9) && ($cnt_files != $limit); $jj++)
        {
                $path=$shop_root.'img/p/'.$ii.'/'.$jj;
                delImage($path);
                for($kk=0; ($kk<=9) && ($cnt_files != $limit); $kk++)
                {
                        $path=$shop_root.'img/p/'.$ii.'/'.$jj.'/'.$kk;
                        delImage($path);
                        for($ll=0; ($ll<=9) && ($cnt_files != $limit); $ll++)
                        {
                                $path=$shop_root.'img/p/'.$ii.'/'.$jj.'/'.$kk.'/'.$ll;
                                delImage($path);
                                for($mm=0; ($mm<=9) && ($cnt_files != $limit); $mm++)
                                {
                                        $path=$shop_root.'img/p/'.$ii.'/'.$jj.'/'.$kk.'/'.$ll.'/'.$mm;
                                        delImage($path);
                                        for($nn=0; ($nn<=9) && ($cnt_files != $limit); $nn++)
                                        {
                                                $path=$shop_root.'img/p/'.$ii.'/'.$jj.'/'.$kk.'/'.$ll.'/'.$mm.'/'.$nn;
                                                delImage($path);
                                        }
                                }
                        }
                }
        }
}
echo 'files: '.$cnt_files.' checked: '.$cnt_checked.' not_found: '.$cnt_not_found.' found: '.$cnt_found."\n";

function delImage($imageDir)
{
        global $limit, $pdo, $cnt_files, $cnt_checked, $cnt_not_found, $cnt_found;
        if ($handle = @opendir($imageDir)) {                    //@ is wriiten to avoid warning message and is handled in else condition
                //echo $imageDir."\n";
                while ($cnt_files != $limit && false !== ($entry = readdir($handle))) {
                        if ($entry != "." && $entry != "..") {
                                $cnt_files++;
                                $pi = explode('-',$entry);
                                if($pi[0]>0 && !empty($pi[1])) {
                                        $cnt_checked++;
                                        if(!checkExistsDb($pdo,$pi[0])) {
                                                $cnt_not_found++;
                                                echo 'rm '.$imageDir.'/'.$entry."\n";
                                                unlink($imageDir.'/'.$entry);
                                        } else {
                                                $cnt_found++;
                                        }
                                } else if (substr($entry, -4) === '.jpg') {
                                        $pi = explode('.',$entry);
                                        $cnt_checked++;
                                        if(!checkExistsDb($pdo,$pi[0])) {
                                                $cnt_not_found++;
                                                echo 'rm '.$imageDir.'/'.$entry."\n";
                                                unlink($imageDir.'/'.$entry);
                                        } else {
                                                $cnt_found++;
                                        }
                                }
                        }
                }
                        closedir($handle);
        }
        else
        {
                //echo $imageDir." doesn't exist\n";
        }

}
 
function checkExistsDb($pdo, $id_image) {
        $r=$pdo->query($q='select \'ok\' ok, (select id_image from ps_image where id_image = '.(int)$id_image.') id_image');
        $row=$r->fetch();
        if($row['ok']!='ok') die( 'Problem with query, please correct');
        return $row['id_image']?true:false;
}

 

Edited by Jesus Ayala (see edit history)
Link to comment
Share on other sites
  • 5 months later...
Jesusaps Newbie

Jesusaps

Posted
Posted

Thank you to all. It's so hard for me to say that I don't know how to put the value:

$shop_root='/var/www/html/'

Tried the last script of Jesus Ayala with:

$shop_root='mastil-boom.es/var/www/html/'

$shop_root='mastil-boom.es/httpdocs/var/www/html/'

$shop_root='www.mastil-boom.es/httpdocs/var/www/html/'

$shop_root='http://www.mastil-boom.es/httpdocs/var/www/html/'

$shop_root='https://www.mastil-boom.es/httpdocs/var/www/html/'

and it doesn't work.

Sorry.

Thank you.

Link to comment
Share on other sites
SAKSCM Rising Star

SAKSCM

Posted
Posted
On 5/31/2021 at 3:56 PM, Jesus Ayala said:

I know this post is old. But the script worked for me. Here it is with some modifications to support two more levels of folders.
  

<?php
// root path of the shop
$shop_root='/var/www/html/';
// limit number of image files to check, set to 10 for testing
$limit=100000000;
 
 
//include $shop_root . '/config/settings.inc.php';
$pdo = new PDO( 'mysql:host=localhost;dbname=DBNAME', 'USER', 'PASSWORD' );
$pdo->setAttribute(PDO::ATTR_ERRMODE, PDO::ERRMODE_EXCEPTION);
$r=$pdo->query('select count(1) cnt from ps_image')->fetch();
echo 'count images database: '.$r['cnt'] . "\n";
 
// reset some counters
$cnt_files=0;
$cnt_checked=0;
$cnt_not_found=0;
$cnt_found=0;

for($ii=1; ($ii<=9) && ($cnt_files != $limit); $ii++)
{
        $path=$shop_root.'img/p/'.$ii;
        delImage($path);
        for($jj=0; ($jj<=9) && ($cnt_files != $limit); $jj++)
        {
                $path=$shop_root.'img/p/'.$ii.'/'.$jj;
                delImage($path);
                for($kk=0; ($kk<=9) && ($cnt_files != $limit); $kk++)
                {
                        $path=$shop_root.'img/p/'.$ii.'/'.$jj.'/'.$kk;
                        delImage($path);
                        for($ll=0; ($ll<=9) && ($cnt_files != $limit); $ll++)
                        {
                                $path=$shop_root.'img/p/'.$ii.'/'.$jj.'/'.$kk.'/'.$ll;
                                delImage($path);
                                for($mm=0; ($mm<=9) && ($cnt_files != $limit); $mm++)
                                {
                                        $path=$shop_root.'img/p/'.$ii.'/'.$jj.'/'.$kk.'/'.$ll.'/'.$mm;
                                        delImage($path);
                                        for($nn=0; ($nn<=9) && ($cnt_files != $limit); $nn++)
                                        {
                                                $path=$shop_root.'img/p/'.$ii.'/'.$jj.'/'.$kk.'/'.$ll.'/'.$mm.'/'.$nn;
                                                delImage($path);
                                        }
                                }
                        }
                }
        }
}
echo 'files: '.$cnt_files.' checked: '.$cnt_checked.' not_found: '.$cnt_not_found.' found: '.$cnt_found."\n";

function delImage($imageDir)
{
        global $limit, $pdo, $cnt_files, $cnt_checked, $cnt_not_found, $cnt_found;
        if ($handle = @opendir($imageDir)) {                    //@ is wriiten to avoid warning message and is handled in else condition
                //echo $imageDir."\n";
                while ($cnt_files != $limit && false !== ($entry = readdir($handle))) {
                        if ($entry != "." && $entry != "..") {
                                $cnt_files++;
                                $pi = explode('-',$entry);
                                if($pi[0]>0 && !empty($pi[1])) {
                                        $cnt_checked++;
                                        if(!checkExistsDb($pdo,$pi[0])) {
                                                $cnt_not_found++;
                                                echo 'rm '.$imageDir.'/'.$entry."\n";
                                                unlink($imageDir.'/'.$entry);
                                        } else {
                                                $cnt_found++;
                                        }
                                } else if (substr($entry, -4) === '.jpg') {
                                        $pi = explode('.',$entry);
                                        $cnt_checked++;
                                        if(!checkExistsDb($pdo,$pi[0])) {
                                                $cnt_not_found++;
                                                echo 'rm '.$imageDir.'/'.$entry."\n";
                                                unlink($imageDir.'/'.$entry);
                                        } else {
                                                $cnt_found++;
                                        }
                                }
                        }
                }
                        closedir($handle);
        }
        else
        {
                //echo $imageDir." doesn't exist\n";
        }

}
 
function checkExistsDb($pdo, $id_image) {
        $r=$pdo->query($q='select \'ok\' ok, (select id_image from ps_image where id_image = '.(int)$id_image.') id_image');
        $row=$r->fetch();
        if($row['ok']!='ok') die( 'Problem with query, please correct');
        return $row['id_image']?true:false;
}

 

Hi, 

Thank you for sharing.

Is this code working in version 1.7.x?

Thank you in advance

  • Like 1
Link to comment
Share on other sites
bnadauld Enthusiast

bnadauld

Posted
Posted (edited)

wow this is still an issue from 2013!!??

is prestashopmanager the way to do it - can it be done using the trial. My img dir is 2gb!

im confused as to what regen thumbnails actually does? does it not tidy up the img directory?

 

NOTE:i found this - gonna give it a try https://boutique.comonsoft.com/gb/free-prestashop-modules/27-regeneration-of-prestashop-images.html

Edited by bnadauld (see edit history)
Link to comment
Share on other sites
SAKSCM Rising Star

SAKSCM

Posted
Posted
4 hours ago, bnadauld said:

wow this is still an issue from 2013!!??

is prestashopmanager the way to do it - can it be done using the trial. My img dir is 2gb!

im confused as to what regen thumbnails actually does? does it not tidy up the img directory?

 

NOTE:i found this - gonna give it a try https://boutique.comonsoft.com/gb/free-prestashop-modules/27-regeneration-of-prestashop-images.html

Yes, it is, my imagen dir is almsot 9gb :(

I have been reading what this module does and it is not the solution only helps with the miniature image regeneration not deleting product images not used anymore 😔

Link to comment
Share on other sites
bnadauld Enthusiast

bnadauld

Posted
Posted
1 minute ago, LightSakura said:

Yes, it is, my imagen dir is almsot 9gb :(

I have been reading what this module does and it is not the solution only helps with the miniature image regeneration not deleting product images not used anymore 😔

 

oh dear - sorry. I just dont get y PS has to create directory after directory in img?

Link to comment
Share on other sites
SAKSCM Rising Star

SAKSCM

Posted
Posted
4 minutes ago, bnadauld said:

oh dear - sorry. I just dont get y PS has to create directory after directory in img?

Looks like prestashop only deletes the miniatures but not the main photo you upload, but i think it doesn't deleted even miniatures... We deleted more or less same amount of products than the amount we created and this folder keeps growing and growing

Link to comment
Share on other sites
SAKSCM Rising Star

SAKSCM

Posted
Posted
6 minutes ago, bnadauld said:

oh dear - sorry. I just dont get y PS has to create directory after directory in img?

I don't know what photoshop is doing but I think yes some directories looks infinite opening folders in image folder, but i think is also the main image you upload.. And the images we upload are not bigger than 1mb we have bellow 5000 productos since we stated the store one year ago we delete almost same amount of products that the amount we created but the directory keeps growing so i believe prestashop is not deleting images

Link to comment
Share on other sites
bnadauld Enthusiast

bnadauld

Posted
Posted (edited)
9 minutes ago, Daresh said:

Because otherwise it would end up with one directory having millions of files, like it was in the old versions.

so every time you do an import via csv with a new product it creates a new child directories in img ...  with most of the directories being empty?

 

this wouldn't be an issue (unless its slowing my site down searching though them all on a shared hosting??) but i use a sql script to build custom delivery forms and my phpadmin cuts off the image location at 20 characters

Untitled picture.png

Edited by bnadauld (see edit history)
  • Like 1
Link to comment
Share on other sites
Daresh Proficient

Daresh

Posted
Posted

You can try to verify it. Check how the images table in the database looks like, pick some image_id that is not there, and then look in the img/p folder for it.

For example if you pick ID 123, you need to see if you have img/p/1/2/3/123.jpg

  • Like 1
Link to comment
Share on other sites
Daresh Proficient

Daresh

Posted
Posted
1 minute ago, bnadauld said:

so every time you do an import via csv with a new product it creates a new child directories in img ...  with most of the directories being empty?

No, the empty directory is there because the image got deleted. But the directories are not deleted.

  • Like 2
Link to comment
Share on other sites
SAKSCM Rising Star

SAKSCM

Posted
Posted (edited)
5 minutes ago, bnadauld said:

so every time you do an import via csv with a new product it creates a new child directories in img ...  with most of the directories being empty?

Untitled picture.png

Actually most of our products are created manually, we have done a few import through csv but the deleting and creating (daily) is manually but yes! There are a lot of directories that are empty 

 

4 minutes ago, Daresh said:

You can try to verify it. Check how the images table in the database looks like, pick some image_id that is not there, and then look in the img/p folder for it.

For example if you pick ID 123, you need to see if you have img/p/1/2/3/123.jpg

Yes exactly but i found out that there are some folders that the id and image are not matching with the producto of the id or there are folders that y the id is not existing 

Edited by LightSakura (see edit history)
  • Like 1
Link to comment
Share on other sites
SAKSCM Rising Star

SAKSCM

Posted
Posted
2 minutes ago, Daresh said:

So it's not easy to tell exactly without a script that would check unused images.

And the script posted in this topic would work for ps1.7? i want to think yes beacuse it was updated a few months ago, but i am not sure if this solve the problem or it is for other thing

On 5/31/2021 at 3:56 PM, Jesus Ayala said:

I know this post is old. But the script worked for me. Here it is with some modifications to support two more levels of folders.
  

<?php
// root path of the shop
$shop_root='/var/www/html/';
// limit number of image files to check, set to 10 for testing
$limit=100000000;
 
 
//include $shop_root . '/config/settings.inc.php';
$pdo = new PDO( 'mysql:host=localhost;dbname=DBNAME', 'USER', 'PASSWORD' );
$pdo->setAttribute(PDO::ATTR_ERRMODE, PDO::ERRMODE_EXCEPTION);
$r=$pdo->query('select count(1) cnt from ps_image')->fetch();
echo 'count images database: '.$r['cnt'] . "\n";
 
// reset some counters
$cnt_files=0;
$cnt_checked=0;
$cnt_not_found=0;
$cnt_found=0;

for($ii=1; ($ii<=9) && ($cnt_files != $limit); $ii++)
{
        $path=$shop_root.'img/p/'.$ii;
        delImage($path);
        for($jj=0; ($jj<=9) && ($cnt_files != $limit); $jj++)
        {
                $path=$shop_root.'img/p/'.$ii.'/'.$jj;
                delImage($path);
                for($kk=0; ($kk<=9) && ($cnt_files != $limit); $kk++)
                {
                        $path=$shop_root.'img/p/'.$ii.'/'.$jj.'/'.$kk;
                        delImage($path);
                        for($ll=0; ($ll<=9) && ($cnt_files != $limit); $ll++)
                        {
                                $path=$shop_root.'img/p/'.$ii.'/'.$jj.'/'.$kk.'/'.$ll;
                                delImage($path);
                                for($mm=0; ($mm<=9) && ($cnt_files != $limit); $mm++)
                                {
                                        $path=$shop_root.'img/p/'.$ii.'/'.$jj.'/'.$kk.'/'.$ll.'/'.$mm;
                                        delImage($path);
                                        for($nn=0; ($nn<=9) && ($cnt_files != $limit); $nn++)
                                        {
                                                $path=$shop_root.'img/p/'.$ii.'/'.$jj.'/'.$kk.'/'.$ll.'/'.$mm.'/'.$nn;
                                                delImage($path);
                                        }
                                }
                        }
                }
        }
}
echo 'files: '.$cnt_files.' checked: '.$cnt_checked.' not_found: '.$cnt_not_found.' found: '.$cnt_found."\n";

function delImage($imageDir)
{
        global $limit, $pdo, $cnt_files, $cnt_checked, $cnt_not_found, $cnt_found;
        if ($handle = @opendir($imageDir)) {                    //@ is wriiten to avoid warning message and is handled in else condition
                //echo $imageDir."\n";
                while ($cnt_files != $limit && false !== ($entry = readdir($handle))) {
                        if ($entry != "." && $entry != "..") {
                                $cnt_files++;
                                $pi = explode('-',$entry);
                                if($pi[0]>0 && !empty($pi[1])) {
                                        $cnt_checked++;
                                        if(!checkExistsDb($pdo,$pi[0])) {
                                                $cnt_not_found++;
                                                echo 'rm '.$imageDir.'/'.$entry."\n";
                                                unlink($imageDir.'/'.$entry);
                                        } else {
                                                $cnt_found++;
                                        }
                                } else if (substr($entry, -4) === '.jpg') {
                                        $pi = explode('.',$entry);
                                        $cnt_checked++;
                                        if(!checkExistsDb($pdo,$pi[0])) {
                                                $cnt_not_found++;
                                                echo 'rm '.$imageDir.'/'.$entry."\n";
                                                unlink($imageDir.'/'.$entry);
                                        } else {
                                                $cnt_found++;
                                        }
                                }
                        }
                }
                        closedir($handle);
        }
        else
        {
                //echo $imageDir." doesn't exist\n";
        }

}
 
function checkExistsDb($pdo, $id_image) {
        $r=$pdo->query($q='select \'ok\' ok, (select id_image from ps_image where id_image = '.(int)$id_image.') id_image');
        $row=$r->fetch();
        if($row['ok']!='ok') die( 'Problem with query, please correct');
        return $row['id_image']?true:false;
}

 

 

Link to comment
Share on other sites
  • 4 months later...

Create an account or sign in to comment

You need to be a member in order to leave a comment

Create an account

Sign up for a new account in our community. It's easy!

Register a new account

Sign in

Already have an account? Sign in here.

Sign In Now
 Share
Go to topic listing
×
×
  • Create New...