Node.js | 웹 크롤링 API 서버 만들기(Windows) | Selenium Crawling Server in Windows

Node.js | 웹 크롤링 API 서버 만들기(Windows) | Selenium Crawling Server in Windows | 테스트 모듈 적용(jest)

개발자티포 2022. 8. 2. 15:27

2022. 8. 2. 15:27

728x90

1. jest 모듈을 설치한다.

$ npm i jest @types/jest ts-jest

2. jest 설정파일을 만든다.

/jest.config.ts

module.exports = {
  "testMatch": [
    "**/__tests__/**/*.+(ts|tsx|js)",
    "**/?(*.)+(spec|test).+(ts|tsx|js)"
  ],
  "transform": {
    "^.+\\.(ts|tsx)$": "ts-jest"
  },
}

3. pakage.json 에 테스트 script를 수정해준다.

  "scripts": {
    "test": "jest",
    "start": "node dist/app.js",
    "build": "tsc -p .",
    "dev": "nodemon --watch \"src/**/*.ts\" --exec \"ts-node\" src/app.ts"
  },

4. test 파일을 만들고 아래처럼 테스트 파일을 만들어준다.

/tests/routers/one.test.ts

// dotenv
import dotenv from 'dotenv'
dotenv.config();

// modules
import { By, until, WebDriver } from 'selenium-webdriver';

// logger
import { logger } from '../../config/winston';

// node-json-db
import { db } from '../../config/nodejsondb';

import { 
    getDriverHandler,

    alertCloseAccept,
    alertCloseDismiss,
    promptCloseHandler,

    addCookie,
    getOneCookie,
    getAllCookie,
    deleteOneCookie,
    deleteAllCookie,

    fileRegister,
    
    findElementById,
    findElementByName,
    findElementByXpath,
    findElementByCss,

    JqChangeValueByID,
    JqRemoveAttribute,

    naviGet,
    naviBack,
    naviForward,
    naviRefresh,

    popupClose

 } from '../../modules';

 describe('GET /one', () => {
    // 웹드라이버 설정
    let driver : WebDriver;

    beforeAll(async () => {
        driver = await getDriverHandler();
    })

    afterAll(async () => {
        await driver.quit()
    })

    test('url 잘 뜨는지 확인', async () => {
        try {
            // 브라우저에 접속
            await driver.get('https://typo.tistory.com/');

            // 현재 주소 가져오기
            const text = await driver.getCurrentUrl();

            // test
            expect(text).toEqual('https://typo.tistory.com/')
        }
        catch(Err) {
            console.log(Err)
            logger.debug(Err)
            throw Error;
        }
    })

 })

/tests/middlewares/http.test.ts

import express, { Request, Response, NextFunction } from 'express';
import {   
    loggerInfo,
    loggerError,
    loggerHttp,
    loggerDebug,
 } from '../../config/winston'
import { httpLoggingMiddleware } from '../../middlewares';

describe('MiddleWare http', () => {
    let mockRequest : Partial<Request>;
    let mockResponse : Partial<Response>;
    let nextFunction: NextFunction = jest.fn();

    beforeEach(()=> {
        mockRequest = {};
        mockResponse = {
            json: jest.fn()
        };
    });

    test('http 로깅 미들웨어 테스트', async () => {
        try {
            await httpLoggingMiddleware(mockRequest as Request, mockResponse as Response, nextFunction);
            
            // nextFunction 까지 잘되는지 확인
            expect(nextFunction).toBeCalledTimes(1);
        }
        catch (Err) {
            loggerDebug.info(JSON.stringify(Err))
        }
    })

})

reqest, response, next 를 mock으로 만들어 테스트해준다.

/tests/middlewares/timeInterval.ts

import express, { Request, Response, NextFunction } from 'express';
import {   
    loggerInfo,
    loggerError,
    loggerHttp,
    loggerDebug,
 } from '../../config/winston'

import { timeIntervalMiddleware } from '../../middlewares';

describe('MiddleWare timeInterval', () => {
    let mockRequest : Partial<Request>;
    let mockResponse : Partial<Response>;
    let nextFunction: NextFunction = jest.fn();

    beforeEach(()=> {
        mockRequest = { originalUrl : '/one' };
        mockResponse = {
            json: jest.fn()
        };
    });

    test('timeInterval 로깅 미들웨어 테스트', async () => {

        try {
            await timeIntervalMiddleware(mockRequest as Request, mockResponse as Response, nextFunction);

            // nextFunction 까지 잘되는지 확인
            expect(nextFunction).toBeCalledTimes(1);
        }
        catch (Err) {
            loggerDebug.info(JSON.stringify(Err))
        }
    })

})

/tests/modules/db/expireHandler.ts

import express, { Request, Response, NextFunction } from 'express';

// node-json-db
import { db } from '../../../config/nodejsondb';

// logger
import { loggerDebug } from '../../../config/winston';
import { expireHandler } from '../../../modules';

describe('Module expireHandler', () => {
    let dbPath : string;

    beforeEach(()=> {
        dbPath = '/one'
    });

    test('30분 지난 데이터 삭제 및 확인', async () => {
        try {
            let nowTime = new Date(); // 현재 시간

            // 30분 지난 데이터 삭제
            await expireHandler(dbPath);

            // 데이터 베이스 안 데이터들
            const datalist = await db.getData(dbPath);

            let index = 0; // 기준점이 되는 index

            // 데이터 매핑 및 시간 차이 계산해서 30분 지난 데이터는 삭제
            await datalist.map(async (item : any, index2: number) => {
                let beforeTime =  new Date(item.date); // 데이터들 저장된 시간
                let diff = (nowTime.getTime() - beforeTime.getTime()) / (1000*60); // 데이터들 시간 차이
                // 저장된 지 30분이 지나면 index 체크
                if(diff > 30) {
                    index = index2;
                }
            })

            // ----- 30분 지난 데이터 없는지 확인 -----
            expect(index).toEqual(0);

        }
        catch(Err) {
            console.log("Err : ",Err)
            loggerDebug.info(JSON.stringify(Err))
        }
    })
})

/tests/modules/selenium/alertHandler.test.ts

// dotenv
import dotenv from 'dotenv'
dotenv.config();

// modules
import { By, until, WebDriver } from 'selenium-webdriver';

// logger
import {   
    loggerDebug,
} from '../../../config/winston';

import { 
    getDriverHandler,
    findElementById,

} from '../../../modules';

 // window type 선언
declare const window: typeof globalThis;

 describe('Module alertHandler', () => {
    // 웹드라이버 설정
    let driver : WebDriver;

    beforeEach(async () => {
        driver = await getDriverHandler();
        await driver.get('https://testpages.herokuapp.com/styled/alerts/alert-test.html')
    })

    afterEach(async () => {
        await driver.quit()
    })

    test('alert 테스트 ( alert 텍스트가 잘 나오는지 )', async () => {
        try {
            await (await findElementById(driver,'alertexamples')).click();

            // alert 창 뜰 때까지 기다림
            await driver.wait(until.alertIsPresent());

            // alert 로 드라이버 이동
            let alert = await driver.switchTo().alert();

            // alertText
            let alertText = await alert.getText();

            // 확인버튼 클릭
            await alert.accept();

            // 다시 원래 컨텐츠로 드라이버 이동
            await driver.switchTo().defaultContent();

            expect(alertText).toEqual('I am an alert box!');
        }
        catch(Err) {
            loggerDebug.info(JSON.stringify(Err))
        }
    })

    test('prompt 테스트', async () => {
        try {
            await (await findElementById(driver,'promptexample')).click();

            // alert 창 뜰 때까지 기다림
            await driver.wait(until.alertIsPresent());

            // alert 로 드라이버 이동
            let alert = await driver.switchTo().alert();

            // alert text 사용하는 곳
            let alertText = await alert.getText();

            // 확인버튼 클릭
            await alert.accept();

            // 다시 원래 컨텐츠로 드라이버 이동
            await driver.switchTo().defaultContent();

            expect(alertText).toEqual('I prompt you');

        }
        catch(Err) {
            loggerDebug.info(JSON.stringify(Err))
        }
    })
})

/tests/modules/selenium/cookieHandler.test.ts

// dotenv
import dotenv from 'dotenv'
dotenv.config();

// modules
import { By, until, WebDriver } from 'selenium-webdriver';

// logger
import {   
    loggerDebug,
} from '../../../config/winston';

import { 
    getDriverHandler,

    addCookie,
    getOneCookie,
    getAllCookie,
    deleteOneCookie,

    findElementById,
    findElementByName,

 } from '../../../modules';
  

 describe('Module cookieHandler', () => {
    // 웹드라이버 설정
    let driver : WebDriver;

    beforeEach(async () => {
        driver = await getDriverHandler();
        await driver.get('https://testpages.herokuapp.com/styled/cookies/adminlogin.html');

        // 로그인 및 쿠키생성
        await (await findElementByName(driver,'username')).sendKeys('Admin');
        await (await findElementByName(driver,'password')).sendKeys('AdminPass');
        await (await findElementById(driver,'login')).click();
    },15000)

    afterEach(async () => {
        await driver.quit()
    })

    test('쿠키 추가하기', async () => {
        try {
            // 쿠키추가
            await addCookie(driver,{name : "name", value : "teepo"});

            // 쿠키 가져오기
            const cookies = (await driver.manage().getCookie('name')).value;

            // 쿠키 확인
            expect(cookies).toEqual('teepo');
        }
        catch(Err) {
            loggerDebug.info(JSON.stringify(Err))
        }
    })

    test('쿠키 하나 가져오기', async () => {
        try {
            // 쿠키 가져오기
            const result = await getOneCookie(driver,'loggedin');

            // 쿠키 확인
            expect(result.value).toEqual('Admin');
        }
        catch(Err) {
            loggerDebug.info(JSON.stringify(Err))
        }
    })

    test('쿠키 전부 가져오기', async () => {
        try {
            // 쿠키 가져오기
            const result = await getAllCookie(driver);

            // 쿠키 확인
            expect(result[0].value).toEqual('Admin');
        }
        catch(Err) {
            loggerDebug.info(JSON.stringify(Err))
        }
    })

    test('쿠키 하나 지우기', async () => {
        try {
            // 쿠키 삭제하기
            await deleteOneCookie(driver,'loggedin');

            // 쿠키 확인
            const result = await getAllCookie(driver);

            // 쿠키가 없어졌는지 확인
            expect(result.length).toEqual(0);
        }
        catch(Err) {
            loggerDebug.info(JSON.stringify(Err))
        }
    })

    test('쿠키 전부 지우기', async () => {
        try {
            // 쿠키 삭제하기
            await deleteOneCookie(driver,'loggedin');

            // 쿠키 확인
            const result = await getAllCookie(driver);

            // 쿠키가 없어졌는지 확인
            expect(result.length).toEqual(0);
        }
        catch(Err) {
            loggerDebug.info(JSON.stringify(Err))
        }

    })

})

/tests/modules/selenium/driverHandler.test.ts

// dotenv
import dotenv from 'dotenv'
dotenv.config();

// modules
import { By, until, WebDriver } from 'selenium-webdriver';

// logger
import {   
    loggerDebug,
 } from '../../../config/winston';

import { 
    getDriverHandler,
 } from '../../../modules';
  

 describe('Module driverHandler', () => {
    // 웹드라이버 설정
    let driver : WebDriver;

    beforeEach(async () => {
        
    })

    afterEach(async () => {
        await driver.quit()
    })

    test('webdriver가 잘 반환되는지 테스트', async () => {
        try {
            driver = await getDriverHandler();
            expect(driver).toBeInstanceOf(WebDriver);
        }
        catch(Err) {
            loggerDebug.info(JSON.stringify(Err))
        }
    })

})

/tests/modules/selenium/fileHandler.test.ts

// dotenv
import dotenv from 'dotenv'
dotenv.config();

// modules
import { By, until, WebDriver } from 'selenium-webdriver';

// logger
import {   
    loggerDebug,
} from '../../../config/winston';

import { 
    fileRegister,

    findElementById,
    findElementByName,

    getDriverHandler,

 } from '../../../modules';
  

 describe('Module fileHandler', () => {
    // 웹드라이버 설정
    let driver : WebDriver;

    beforeEach(async () => {
        driver = await getDriverHandler();
        await driver.get('https://testpages.herokuapp.com/styled/file-upload-test.html')
    })

    afterEach(async () => {
        await driver.quit()
    })

     test('fileRegister', async () => {
         try {
             await fileRegister(driver,By.id('fileinput'),['파일 위치'])
             await (await findElementByName(driver,'fileinput')).click()
             var text = await (await findElementById(driver,'uploadedfilename')).getText()
             expect(text).toEqual('image.jpg')
         }
         catch(Err) {
             loggerDebug.info(JSON.stringify(Err))
         }
     })

})

/tests/modules/selenium/findHandler.test.ts

// dotenv
import dotenv from 'dotenv'
dotenv.config();

// modules
import { By, until, WebDriver } from 'selenium-webdriver';

// logger
import {   
    loggerDebug,
 } from '../../../config/winston';

import { 
    getDriverHandler,

    findElementById,
    findElementsById,
    findElementByName,
    findElementByXpath,

    findElementByClass,
    findElementsByName,
    findElementsByXpath,
    findElementsByClass

 } from '../../../modules';
  

 describe('Module findElementHandler', () => {
    // 웹드라이버 설정
    let driver : WebDriver;

    beforeAll(async () => {
        driver = await getDriverHandler();
        driver.get('https://testpages.herokuapp.com/styled/find-by-playground-test.html')
    })

    afterAll(async () => {
        await driver.quit()
    })

    test('findElementById', async () => {
        try {
            let text = await (await findElementById(driver,'p1')).getText();
            expect(text).toEqual('This is a paragraph text')
        }
        catch(Err) {
            loggerDebug.info(JSON.stringify(Err))
        }
    });

    test('findElementByName', async () => {
        try {
            let text = await (await findElementByName(driver,'pName1')).getText();
            expect(text).toEqual('This is a paragraph text')
        }
        catch(Err) {
            loggerDebug.info(JSON.stringify(Err))
        }
    });

    test('findElementByXpath', async () => {
        try {
            let text = await (await findElementByXpath(driver,'//*[@id="p1"]')).getText();
            expect(text).toEqual('This is a paragraph text')
        }
        catch(Err) {
            loggerDebug.info(JSON.stringify(Err))
        }
    });

    test('findElementByClass', async () => {
        try {
            let text = await (await findElementByClass(driver,'explanation')).getText();
            expect(text).toEqual('This is a set of nested elements. There are various ways to locate each of the elements. Challenge yourself to find as many ways of locating elements as possible.')
        }
        catch(Err) {
            loggerDebug.info(JSON.stringify(Err))
        }
    });

})


describe('Module findElementsHandler', () => {
    // 웹드라이버 설정
    let driver : WebDriver;

    beforeAll(async () => {
        driver = await getDriverHandler();
        await driver.get('https://testpages.herokuapp.com/styled/find-by-playground-test.html')
    })

    afterAll(async () => {
        await driver.close()
    })

    test('findElementsById', async () => {
        try {
            let text = await (await findElementsById(driver,'p1'))[0].getText();
            expect(text).toEqual('This is a paragraph text')
        }
        catch(Err) {
            loggerDebug.info(JSON.stringify(Err))
        }
    });

    test('findElementsByName', async () => {
        try {
            let text = await (await findElementsByName(driver,'pName1'))[0].getText();
            expect(text).toEqual('This is a paragraph text')
        }
        catch(Err) {
            loggerDebug.info(JSON.stringify(Err))
        }
    });

    test('findElementsByXpath', async () => {
        try {
            let text = await (await findElementsByXpath(driver,'//*[@id="p1"]'))[0].getText();
            expect(text).toEqual('This is a paragraph text')
        }
        catch(Err) {
            loggerDebug.info(JSON.stringify(Err))
        }
    });

    test('findElementsByClass', async () => {
        try {
            let text = await (await findElementsByClass(driver,'explanation'))[0].getText();
            expect(text).toEqual('This is a set of nested elements. There are various ways to locate each of the elements. Challenge yourself to find as many ways of locating elements as possible.');
        }
        catch(Err) {
            loggerDebug.info(JSON.stringify(Err))
        }
        });
})

/tests/modules/selenium/jqueryHandler.test.ts

// dotenv
import dotenv from 'dotenv'
dotenv.config();

// modules
import { By, until, WebDriver } from 'selenium-webdriver';

// logger
import {   
    loggerDebug,
 } from '../../../config/winston';

// node-json-db
// import { db } from '../../config/nodejsondb';

import { 
    getDriverHandler,

    findElementById,

    JqChangeValueByID,
    JqSetAttribute,
    JqRemoveAttribute,
 } from '../../../modules';
  

 describe('Module jqueryHandler', () => {
    // 웹드라이버 설정
    let driver : WebDriver;

    beforeEach(async () => {
        driver = await getDriverHandler();
        await driver.get('https://testpages.herokuapp.com/styled/html5-form-test.html')
    })

    afterEach(async () => {
        await driver.quit()
    })

    test('JqChangeValueByID', async () => {
        try {
            let value = "teepo";
            await (await findElementById(driver,'email-field')).clear()
            await JqChangeValueByID(driver,'email-field',value);
            let text = await (await findElementById(driver,'email-field')).getAttribute('value');
            expect(text).toEqual(value)
        }
        catch(Err) {
            loggerDebug.info(JSON.stringify(Err))
        }
    })

    test('JqSetAttribute', async () => {
        try {
            await JqSetAttribute(driver,'email-field',['disabled','true']);
            let bool = await (await findElementById(driver,'email-field')).getAttribute('disabled');
            expect(bool).toBe("true")
        }
        catch(Err) {
            loggerDebug.info(JSON.stringify(Err))
        }
    })

    test('JqRemoveAttribute', async () => {
        try {
            await JqSetAttribute(driver,'email-field',['disabled','true']);
            await JqRemoveAttribute(driver,'email-field','disabled');
            let bool = await (await findElementById(driver,'email-field')).getAttribute('disabled');
            expect(bool).toBeNull()
        }
        catch(Err) {
            loggerDebug.info(JSON.stringify(Err))
        }
    })

})

/tests/modules/selenium/navigationHandler.test.ts

// dotenv
import dotenv from 'dotenv'
dotenv.config();

// modules
import { By, until, WebDriver } from 'selenium-webdriver';

// logger
import {   
    loggerDebug,
 } from '../../../config/winston';

// node-json-db
// import { db } from '../../config/nodejsondb';

import { 
    getDriverHandler,

    findElementById,

    naviGet,
    naviBack,
    naviForward,
    naviRefresh,

 } from '../../../modules';
  

 describe('Module navigation', () => {
    // 웹드라이버 설정
    let driver : WebDriver;
    let url = 'https://testpages.herokuapp.com/styled/index.html';

    beforeEach(async () => {
        driver = await getDriverHandler();
        await naviGet(driver,url)
    })

    afterEach(async () => {
        await driver.quit()
    })

    test('naviGet', async () => {
        try {
            let currentUrl = await driver.getCurrentUrl();
            expect(currentUrl).toEqual(url);
        }
        catch(Err) {
            loggerDebug.info(JSON.stringify(Err))
        }
    })

    test('naviBack', async () => {
        try {
            await (await findElementById(driver,'basicpagetest')).click();
            await naviBack(driver);
            let currentUrl = await driver.getCurrentUrl();
            expect(currentUrl).toEqual(url);
        }
        catch(Err) {
            loggerDebug.info(JSON.stringify(Err))
        }
    })

    test('naviForward', async () => {
        try {
            await (await findElementById(driver,'basicpagetest')).click();
            await naviBack(driver);
            await naviForward(driver);
            let currentUrl = await driver.getCurrentUrl();
            expect(currentUrl).toEqual('https://testpages.herokuapp.com/styled/basic-web-page-test.html');
        }
        catch(Err) {
            loggerDebug.info(JSON.stringify(Err))
        }

    })

    test('naviRefresh', async () => {
        try {
            await naviRefresh(driver);
            let currentUrl = await driver.getCurrentUrl();
            expect(currentUrl).toEqual(url);
        }
        catch(Err) {
            loggerDebug.info(JSON.stringify(Err))
        }

    })

})

/tests/modules/selenium/popupHandler.test.ts

// dotenv
import dotenv from 'dotenv'
dotenv.config();

// modules
import { By, until, WebDriver } from 'selenium-webdriver';

// logger
import {   
    loggerDebug,
 } from '../../../config/winston';

import { 
    getDriverHandler,

    findElementById,

    naviGet,

    popupClose

 } from '../../../modules';
  

 describe('Module popupHandler', () => {
    // 웹드라이버 설정
    let driver : WebDriver;

    beforeEach(async () => {
        driver = await getDriverHandler();
        await naviGet(driver,'https://testpages.herokuapp.com/styled/alerts/alert-test.html');
    })

    afterEach(async () => {
        await driver.quit()
    })

    test('popupClose', async () => {
        try {
            await (await findElementById(driver,'promptexample')).click();
            popupClose(driver);
            let windowNmber = await (await driver.getAllWindowHandles()).length;
            expect(windowNmber).toEqual(1);
        }
        catch (Err) {
            loggerDebug.info(JSON.stringify(Err))
        }
    })

})

5. 빌드하고 테스트해본다.

$ npm run build
$ npm run test

728x90

'Back-End > Node.js' 카테고리의 다른 글

Node.js \| 웹 크롤링 API 서버 만들기(Windows) \| Selenium Crawling Server in Windows \| 모듈화 하기 (0)	2022.08.01
Node.js \| 웹 크롤링 API 서버 만들기(Windows) \| Selenium Crawling Server in Windows \| 결과값 저장하기 (0)	2022.07.29
Node.js \| 웹 크롤링 API 서버 만들기(Windows) \| Selenium Crawling Server in Windows \| Selenium 시작하기 (0)	2022.07.29
Node.js \| 웹 크롤링 API 서버 만들기(Windows) \| Selenium Crawling Server in Windows \| 환경세팅 (0)	2022.07.29
Node.js \| winston으로 로깅(Logging) 미들웨어 만들기 (2) (0)	2022.05.26

티포의개발일지